Okay. I guess I'm gonna make this account a little more annoying. I'm gonna spend a little time each week exploring my mastodon experience and talking about how to make it better.
I say "annoying" because I'd rather be interesting than just useful. And I think this kind of content is the opposite. Potentially useful but overall not the most interesting. (It's okay if you disagree. We don't have to talk about it)
https://social.polotek.net/@polotek/111966798276024291
Just for some context. I run my own mastodon instance with just one user. Me. I rent hosting from https://masto.host/. I have done very little maintenance or configuration so far. But here is a brief list:
- I customized my domain
- I turned on the (paid) search feature from masto.host
- Some random things during the initial setup wizard. Don't remember, but I'll check
So the topic for this week is performance. I frequently experience very slow loading times for different views. Slow as in multiple seconds. The home timeline, notifications view, single post views with replies, etc.
Most people start by assuming it's my client. I tried the official masto client, Tusky, elk.zone, and now phanpy.social. All have the same issue. So it's probably something on the server side, and thus something I may be able to control as the admin of my instance.
I'm gonna do some exploring and experimenting. I'll let you know how it goes. First easy step is try increasing the system resources on my masto.host account. I currently pay for the lowest tier. Strangely enough, masto.host talks about stuff like CPU and storage, but doesn't mention RAM. We'll see.
Posting is also super slow at times. The above post with the two images took around 30 seconds. That is unsustainable.
Interesting. I'm very familiar with sidekiq. Is it on my instance or you mean other instances I deliver to? If it's on my side, I would assume masto.host shares a redis cluster across many(all?) of it's hosted instances. In which case... yikes!
https://cosocial.ca/@timbray/111966910673636528
Quick follow up. I went ahead and smashed the button on masto.host to upgrade to the highest tier for personal instances (as opposed to larger community plans).
The difference is *very* noticeable already. All load times and actions are much faster. So ya know, if you want things to be faster with minimal effort, just pay 4x more money.
I'll do a bit more investigation later this week. I don't think I really want to keep paying $19 per month for this. That can't be the only option.
@polotek I would love to understand this aspect of ActivityPub better. Is it inherent to the protocols, or is this distribution bottleneck particularly bad with Mastodon and Sidekiq? Here’s an older article on how it seems be sort of a self-inflicted DoS platform in some configurations: https://ar.al/2022/11/09/is-the-fediverse-about-to-get-fryed-or-why-every-toot-is-also-a-potential-denial-of-service-attack/
@polotek This piece from jwz about the impact of posting a link to Mastodon is related because of how all the servers generate link previews. https://www.jwz.org/blog/2022/11/mastodon-stampede/
@polotek @timbray your instance would have a redis process + a Sidekiq process with concurrency set by masto.host (4?). If Sidekiq only has 4 threads and you've got 2000 followers, pushing a post can take a few minutes. Your Admin UI should have the Sidekiq UI in it somewhere where you can monitor the queues.
I know you both are experienced devs, here's an open invitation to you or anyone else to my weekly Office Hour. Always happy to chat more. https://sidekiq.org/support.html
@getalifemike @timbray thanks Mike. For what it's worth, I'm a fan of sidekiq. I just know when and where it falls down without putting in some effort. And part of the context of this discourse is about how much effort is required to have a reasonable experience with mastodon. I may take you up on the office hours, but it'll be after I do some exploration on my own.
@polotek I've got a masto.host setup and I like it well enough but I'm afraid I'll want to be able move hosts/server software on the domain and I don't yet understand the consequences of doing that. The only thing I know for sure is I could migrate to a different mastodon host... probably.
@polotek I’ve been wondering about this too… I know mastodon can be really CPU hungry
@polotek i pay $39/month and it still feels slow sometimes
@polotek you are clearly not benefitting from any economies of scale here, and paying a single user instance tax.
@polotek huh interesting and super odd! I'm on an identical setup - masto.host, single user, smallest plan - and I haven't experienced any performance issues. We have a similar number of followers, too, so it's probably not related to that. Wild. Going to be very interested to see what you find out.
@polotek
@jacob with such a disparity, maybe worth looping in @mastohost. I use them as well and know different app and db servers occasionally have issues, if it’s not a configuration thing.
If that is a regular occurrence then we should look into that because it shouldn't be happening. So, please reach out when the issue is present, so we can attempt and get to the bottom of it.
Regarding testing speed, I suggest using the browser if you experience a speed issue as that will remove third party issues (namely apps) as a possible cause.
@mastohost @alec @jacob as I stated above, this happens with multiple clients.
Are there any upgrades to memory with the pricing tiers? Only processing threads and storage are mentioned explicitly. https://social.polotek.net/@polotek/111966916310343916
@mastohost @polotek @alec @jacob I experienced some slowness when I first joined. Moving up to the $9/mo plan resolved it for me, which imho is reasonable.