Lemmy Support

4651 readers

1 users here now

Support / questions about Lemmy.

Matrix Space: #lemmy-space

founded 6 years ago

MODERATORS

dessalines@lemmy.ml

Lemmy Devs: could you shed light on the range of scalability issues we're about to see with the reddit influx? (sopuli.xyz)

submitted 2 years ago* (last edited 2 years ago) by theory@sopuli.xyz to c/lemmy_support@lemmy.ml

34 comments fedilink hide all child comments

I expect lemmy.ml to be hugged to death, but that's just a DoS.

More interestingly would be how the activitypub network reacts under reddit-like loads and behaviour, and as i understand it, things shouldnt be too bad?

Is lemmy scalable?

you are viewing a single comment's thread
view the rest of the comments

[–] Grouchy@lemmy.grouchysysadmin.com 3 points 2 years ago (1 children)

I've tested running multiple instances of pict-rs using distributed storage (Moosefs) for the uploaded file directory. I ran into several weird issues. Mainly that images would not load, or end up broken. But then refresh the page and hit a different pict-rs instance and it worked for some images, but broke for others. So now I run a single pict-rs instance, still on distributed storage, but everything works. This is on the 3x branch. I believe the 4x branch is still in rc stage.

[–] PriorProject@lemmy.world 1 points 2 years ago (1 children)

I've tested running multiple instances of pict-rs using distributed storage [and] ran into several weird issues.

Well that's a bummer.

At the very least, a Lemmy install is comprised of at least 4 discrete processes. I have to think that putting lemmy, lemmy-ui, pict-rs (backed by object-storage to provide scaleable I/O and let the pict-rs host focus on CPU), and PG (optionally with additional read-replicas) on separate boxes would result in a hardware platform that outscales the current codebase for a year or two while they clean up perf issues that crop up with large user/post/comment/community counts.

[–] Grouchy@lemmy.grouchysysadmin.com 2 points 2 years ago

That's basically what I'm doing. A singleton instance for Pict-rs and Postgres. Multiple compute nodes for Lemmy and Lemmy-ui being accessed over load balancers. All of data is placed on distributed storage so I can quickly fail-over Pict-rs or Postgres if needed. It's enough for now, though I don't think it could handle Reddit levels of traffic.