Page 1 of 1

This forum is flooded with bots

Posted: 13 Jan 2024, 01:29
by Porcupine
Starting around maybe 5 years ago, I noticed that unknown bot armies had flooded oratan.com forum. Prior to that time, the average number of guests and registered users at any given time would be 0 or 1 people (which matches the active population of the game). When the bots came, that number often jumped to 30~50, but it would dwindle back down to 9 or so most of the time, and it didn't really affect anything so it wasn't a problem.

These past few months the amount of bot activity seems to have increased by some multiple, and today it's suddenly multiplied yet again. The maximum number of simultaneous users is the same, but the swarms appear and disappear much more often. Now there seem to be leader bots with registered accounts who regularly summon these swarms of 20~40 guest bots. I'm not sure what they do, I think they just read posts, but they are doing it so often today that this site is slowing down. Opening posts and pages, which is usually fairly consistent taking half a second or so, is sluggish and will randomly vary from the typical half second up to 5 seconds.

I guess this isn't a big deal but I'm not so sure these bots help us in any way. The leader bot accounts seem to be Ahrefs [Bot], Bing [Bot], and Google [Bot]. I don't know if it is possible to ban them, or if that would do any good, since it's their unregistered guest bot swarms that are probably the real slowdown factor.

Maybe the guest swarms are not bots, but random people around the world who are clicking on oratan.com links having no idea what Virtual On is, but having come up in a search for some random reason? Hm, this wouldn't explain why the guest swarms typically rapidly increase then decrease in number, though.

edit: A day later, now I see Majestic-12 [Bot] for the first time. Internet search suggests that it's a known malicious attack bot that eats up bandwidth to take down sites.

Re: This forum is flooded with bots

Posted: 13 Jan 2024, 16:49
by MentholMoose
Thanks for the heads up! The traffic causing the problem was from Alibaba Cloud and is now blocked.

Re: This forum is flooded with bots

Posted: 13 Jan 2024, 17:46
by Porcupine
Thanks, everything is working normally again. There are still the same 20~40 guest swarmers, with several named bots, but it's been this way for the past years and generally isn't a problem. Now I see a Baidu [Spider] account logged in. I guess the real problem though was essentially just one or a few unnamed guests causing mass mayhem?

Re: This forum is flooded with bots

Posted: 13 Jan 2024, 20:32
by MentholMoose
I think it was a misconfigured or broken web scraper, although the user agent was randomized which isn't typical of legitimate scraping e.g. by search engines. Thanks for letting me know.

Re: This forum is flooded with bots

Posted: 14 Jan 2024, 02:00
by Porcupine
I'll just use this place to catalog all the registered bot users that I notice. Today I newly see Brandwatch [bot], and Bytespider [Bot]. We are really popular!

Also, after a day the total number of simultaneous unregistered users has dwindled back down to "normal" again, like 9 or so. Plus we typically have like 2 or 3 registered bots on now.

Just as I wrote this, the biggest group of registered bots online together that I have ever seen by far has suddenly appeared: Ahrefs [Bot], Brandwatch [bot], Bytespider [Bot], Petal [Crawler], Pinterest [Bot], Porcupine, Sogou [Crawler]

Plus SinisterSticks, a new regular user who is probably not a bot, but I'll list him for now because who knows what could be going on hehe. I noticed that the bot names are all in bold but people names aren't, not sure how this forum identifies them.

Now I see FriendlyCrawler [Crawler] too, what is this some kind of parade hehe.

Re: This forum is flooded with bots

Posted: 14 Jan 2024, 12:13
by MentholMoose
Yes SinisterSticks is a new user, welcome!

Since I was looking at logs I went ahead and "registered" some of the legitimate bots that weren't previous registered.
Identifying bots is important so that phpBB can serve them content which is more appropriate for search engines - dead links to pages without content are omitted, e.g. posting pages, report post pages etc. Bots never receive a session ID in the URL, which should not appear in the search results.