You might have already noticed that everything from http://kakaostats.alexbueno.me/ now redirects to http://www.kakaostats.net/ .
This will be the definitive domain, but since all over the world the known address is the subdomain to my own domain, I’ll left this redirect for like… forever!
Same will happen with this blog and Forum.
At 21/09/2016 roundly 00:00 UTC time restore was finished!
Took extra time for two reasons:
- Old Atom D2500 motherboard was screwed and it was showing some kernel errors from time to time that froze the OS. One friend gave me an Intel 775 motherboard, it was still in the box never used, so I bought a cheap Core2Duo E8400 plus 4GiB of memory in some kind of eBay we have in Brazil. It’s currently overclocked at 3.6GHz getting close to 90°C because it came without cooler and is relying on a Celeron 430 cooler. I bought a new one already, but did not receive yet.
- Because sometimes there’s no user or data file from Stanford, restore script always got stuck on that and stopped. Of course first Murphy’s law is unmerciful and the first time it happened was during weekend while I was on the beach without internet access. Almost tree days without restore running. Aaaand I sleep from time to time, so happened few times during that either.
I was checking yesterday the source code of KakaoStats everyone knew, and I guess I know how to get it running again. I’ll travel for training today and I’ll have some spare time in hotel today and tomorrow evening so I already started up a virtual machine to play on it.
What do you think? Should I do it? Which version you liked more? Should I get both running? Let me know…!
You might have noticed that since 13/09/2016 @ 15:20 UTC there’s no updates.
At first I was assuming this was just another time without new files. It happens from time to time. Then I connected the server to check and somehow it’s downloading the files and importing correctly without any errors.
Since I was already working on a script to restore the database with backup files in the coming soon Dell PowerEdge, I started doing this in a computer I have here. Pretty simple, it’s an Atom D2500 with 4GB of RAM, but works (or seems so). This one is serving new domain already.
While it started restoring I got back to current server and begin to investigate WTH no new data in the data base. No clue so far!
So I gave up for now, because I have other stuffs to do (like work) and decided to run the restore script in the old server. No data will be lost since even with the script running new files are being downloaded.
I just registered a new domain to KakaoStats, soon I’ll migrate from http://kakaostats.alexbueno.me/ to http://www.kakaostats.net/ . Forum and blog will also be migrated!
Unfortunately someone already registered the .com domain and it’s for sale for something like $10k. No way!
Also I detected some problems in the database after this long period without new files. The interface that download new files wasn’t downloading nothing because it detected that it was still the same file, but it was importing this file hourly.
This seems to create a little mess in the database, some donor names are showing wrong. I’ll begin to implement this checking in the interface to avoid problem, and will restore the database to fix it. Restore script is also needed.
Keep you posted…
What do you want first? Good news or bad news?
Let’s go with chronological order… Good news first!
Last Friday (02/08/2016) I got an email from Ed and he offered a whole new server to host KakaoStats. I wasn’t really expecting that! Really!! When I read the email with a link for the proposed server I was expecting something like a PC (what I was planning to replace the actual laptop), but then I opened the link and became speechless… A Dell PowerEdge R710 with 3x 1TiB HDD (hot swap), 96GiB of RAM and 2x Xeon 6 cores (don’t remember the model), plus everything else that comes in this kind of server. Not even in my hottest dreams I ever imagined a server like this to run KakaoStats!!
Don’t even need to talk about its reliability, but the power of this server opens a whole new world to KakaoStats. I already had the plan to re-write KakaoStats codes from front and backend, and rebuild its DataBase. But now it can be done in a new level, and that’s the plan. Stay tuned for news!!
Aaand the bad news…
Later in the same day someone broke my car to steal the spare tire and also stole both my laptops! Some documents were also lost, some were found, but still no clue of laptops and even my little daughter clothes that I bought couple of hours before that. My HTPC was also there and they didn’t steal, my RayBan glasses, phone charger, 5 liters of 2 stroke oil to my go-kart wasn’t steal also. I’m really sad because, not only I lost my brand new laptop (other one belong to the company I work for), but the contents inside. Most stuff was saved in OneDrive (
piece of shit, don’t use it) but 2 external HDDs with 1TiB was stole and my MP3 folder from 20 years of downloads (some was still downloaded with Napster) was in secondary HDD and I’m not sure if I have backup.
But anyways… Life still goes on!
Just to keep you posted, since 04.09.2016 10:20 there’s no valid team/donor files from Stanford. That’s why Kakao has no new data since then.
So there I was, drinking a beer with a friend of mine and we started talking about KakaoStats. He liked the system itself and bla bla bla, so I told him it’s running on an old laptop and I’m saving some money to buy a proper PC to run it.
After a couple more beers he gave me the idea to ask for donations. I’ve never actually considered that. I decide to run KakaoStats because I like to work with servers and I’ve been missing that lately. (I used to be network/system administrator until I started working with automation) So, it’s kind of a hobby for me…
But unlikely my go-kart hobby, I’m not the one who enjoys it more. Of course I do enjoy it quite a lot, even in the background or checking my points.
Anyway… I took some time, re-activated my Paypal account (and another Brazilian-only payment gateway) and created the donate buttons! Feel free to donate the amount you feel you should donate, small or big values will be only used to improve KakaoStats. All donations are done in BRL, so check the currency!
My first plan is to buy a new hardware. I already have a rack case, PSU and two 2TiB 2.5″ HDD. Plus I found this HDD case yesterday and I’m really mad because I just ordered a new one from China. When arrive I’ll try to sell it and save the money for other hardware.
While I don’t have the money, I’m checking some good cost-benefit CPUs and I found out a bottleneck in the current hardware. The 2TiB HDD I’m using runs only at 5.400 rpm, I have room but I don’t have performance. When importing data and it needs the HDD the CPU usage goes low and it takes time, but the worst part is that navigating in KakaoStats become slower. Indexed part still normal, but sometimes it’s needed to go to disk to fetch some data, and it also is bad for importing task. So I added to hardware list also two SSD drives (will be mirrored) to run the OS and the database and then HDD will be used only for backups.
Please don’t feel obligated to donate money. Kakao will still exists and I’ll also buy some things by myself for it.
There was a change/fix in “Last Update” column.
More information here.
Since I was with “dirty hands” installing this WordPress to keep you posted about Kakao, I also installed a forum so we could interact.
Hi everyone, I created this blog to keep you updated about what’s going on about Kakao like server crashes, maintenance and stuff.
As you may know, I started running Kakao in May 2015 (I guess) and in Jan 2016 server went down for a long period. I was hired a server in a datacenter to run Kakao and was paying with my credit card. Unfortunately someone cloned my credit card while I was in a long trip.
My bank cancelled it and I did not have enough time to backup the contents of the site like database and user/team data files, so I needed to start from zero again.
I was waiting until I get my credit card back to reactivate the site again. But I still don’t have it yet and not sure when I’ll get it back. So I decided to host it on my own. Main idea was to host it in my own internet link, but seems that my ISP is avoiding NAT on the router and this makes me unable to make the server online.
One friend of mine who has some good infrastructure on his office offered me a place on his rack to put my server there. And on 17/08/2016 Kakao was back!!!
Currently Kakao is running on an old laptop I have, it’s a HP DV5 with an Athlon II P320 Dual Core @ 2.1 GHz, 8GB of RAM and 2TB of HDD. It seems weaker then the i7 @ 3.4 GHz and 16 cores, but somehow it’s running faster than before.
There’s three issues on this laptop:
- It’s not 100% reliable. I got it damaged from a friend, send it to repair and used for other purposes for a year. It’s stable but I’m not able to reboot it remotely because it most of the times get stuck on BIOS. This concern me about how long it will survive.
- As the database grows (currently 36GB) I notice it is taking a little more time to import user and team files, this might be a problem in the future.
- It only has one HDD. No mirror. No back-ups. I’m configuring an OneDrive account with 1TB space I have to backup the files. But still, if the HDD suddenly decides to die it will cost me some time to restore everything.
Based on that, I’m planning to replace this laptop for a new server ASAP. My friend who is hosting the server offered me a server case and a PSU, but I still need the “rest”. The total amount of cores doesn’t actually matter since the database only can use one at a time while importing files and the traffic is not big enough to make it use all of them serving pages. Same for Apache. Good news is that it’ll be cheap to buy the mainboard, memory and CPU. Expensive part will be another 2TB HDD to create a mirror.
That’s it for now… Thanks!