This site is currently struggling to handle the amount of new users. I have already upgraded the server, but it will go down regardless if half of Reddit tries to join.
However Lemmy is federated software, meaning you can interact seamlessly with communities on other instances like beehaw.org or lemmy.one. The documentation explains in more detail how this works. Use the instance list to find one where you can register. Then use the Community Browser to find interesting communities. Paste the community url into the search field to follow it.
You can help other Reddit refugees by inviting them to the same Lemmy instance where you joined. This way we can spread the load across many different servers. And users with similar interests will end up together on the same instances. Others on the same instance can also automatically see posts from all the communities that you follow.
Edit: If you moderate a large subreddit, do not link your users directly to lemmy.ml in your announcements. That way the server will only go down sooner.
You must log in or register to comment.
IMHO, selecting an instance is definitely the biggest user experience problem Lemmy has at the moment. New users who are unfamiliar with the platform are going to pick the biggest instances, and that’s going to create performance problems.
We’ll need to prioritize work on instance browsing. Lemmy has outgrown the experience over at join-lemmy.org. If I could wave a magic wand, instance browsing and onboarding would have a way to show instance capacity / performance, a way to categorize and filter instances, and a way to recommend instances based upon interests. That would probably help to spread people out more evenly.
No problem just make a pull request with your changes.
You probably don’t want my code if you want a stable platform. ;)
That said, I dig what y’all are doing, and I’m veteran experience / interaction designer who’s been around the block for a few decades. So I might be able to find some time to mockup some experience concepts and or help to run user tests with audiences that your curious about.
I’m more of a backender myself, but I think some UX mockup would go a long way in getting this improved.
There’s a website I highly recommend called fediverse observer, it doesn’t really go based on interest, but it has some other factors it uses and I really like it.
I think that there should be some meaningful way to “preview” aspects of one instance that may make it more attractive than another instance to a new user. I just joined lemmy.world today simply because it seemed the most generic. Onboarding process could use some work; https://lemmy.world/post/37906 is great at explaining it but people will only really see it for the first time once they join…
Also I have no clue if that second link works. ¯\(ツ)/¯
You might wanna consider temporarily closing sign-up requests on
lemmy.mlsimilarly to howmastodon.socialdid it during its large influx. Making a sign-up request and just receiving an infinite loading icon is a very frustrating experience.Similarly, you want to make it as easy as possible to financially contribute to lemmy, even if it means using proprietary platforms like Patreon.
Overall, the current Reddit API change is probably one of the largest opportunities for lemmy right now, so smoothing over the user experience as fast as possible in the coming days will be of atmost importance if we want lemmy to become a viable Reddit alternative…
Is scaling the server a largely financial issue, or not? @nutomic@lemmy.ml
could you reasonably confidently say that you could 10x the amount of users for something like 1000$/mo on liberapay?
If so, would you mind setting a “goalpost” for the community to help lift the financial burden?I think they said they’re at the highest tier of their provider. May need to migrate to a different provider and get a beefier setup.
Do Lemmy instances not scale horizontally?
They do, but I’m not sure how well, I’m not a dev, and have no programming knowledge, so looking at the documentation looks like arcane hieroglyphs.
I’m pretty sure I read a comment about it from one of the devs, but can’t recall the fine details of the conversation.
In theory, they can. But it depends on how it’s deployed.
From my cursory look at the deployment docs, Lemmy’s default deployment option is via docker. It relies on a postgreSQL server, which may or may not scale horizontally depending on the admin’s choice of implementation. For example, a deployment on AWS using Aurora would theoretically utilize auto-scaling.
I haven’t personally deployed an instance so, grain of salt.
EDIT: A good discussion about DB scaling here: https://github.com/LemmyNet/lemmy/issues/3005
I would be happy to use another instance but my account is on this one. Is there a way to migrate an account, or perhaps “link” accounts on multiple instances somehow?
AFAICT no. There is an open issue on the Lemmy GitHub repo. In general, all ActivityPub services I’ve used have this same account stratification problem.
I think this is key. Have the possibility to move an account to another instance or have it spread out somehow. This would also secure the account in case an instance dies for some reason.
I would appreciate this as well. Besides the flood of users issue, this server’s theme (Marxist-Leninist) doesn’t mesh with my politics. I created it in the early days of Lemmy, so I have an extensive history that I am loath to sacrifice.
Wait isn’t lemmy.ml a general purpose server?
It is, but unlike elsewhere, ML folks are usually not banned or defederated here, only if they break rules (like insulting other users).
@radarsat1 @nutomic I’m still working this out myself, but you can browse Lemmy on any account, and then comment from your mastodon account by searching the commenter (@radarsat1 in the search field did it for me), and then replying to them on mastodon. Pretty cool how it’s linked, plenty of opportunities to build apps that make interacting between servers easier.
I applied for a few other instances but this one came through first. Your downfall is being too good compared to the competition.
I’ll be honest. I only applied for this one but that was because I had (still have) no idea what I’m doing lol
That’s how I wound up here too.
lemmy.ml should be a roundrobin dns that sends you to a random instance in the pool. Or else you will re-centralize lemmy and curmble under the IT bill.
Except (as far as I’m aware) your account only exists on one instance. So, if I end up on beehaw.org due to the round-robin, my account on lemmy.ml will not authenticate to that instance. I would have to have a separate account per instance which is hundreds of accounts.
That’s actually a good idea
For non technical users, the idea of instances can be a very confusing concept (the email analogy is a good one but its still confusing for people). I know you guys have a lot on your plate in terms of development wise, however I hope that prioritizing keeping lemmy.ml up is high up there. I say this because its the instance that most users from Reddit will flock to. And the last thing they need is to create an account then have the site go down for 6 hours. I havent experienced it going down. Although hopefully you have a backup site for when it does (what I mean is just a page that says your down/your working on fixing it… Try these instances instead.)
It’s confusing even for me. Do I get to see every post in every instance? Do I get to see all the comments? Do others get to see all of my posts irrespective of their instance? Can I see and interact with all users irrespective of their instance? Can I browse Lemmy if my instance is overloaded? If not, can I seamlessly move to a different instance?
Do I get to see every post in every instance?
Do I get to see all the comments?
Do others get to see all of my posts irrespective of their instance?
Can I see and interact with all users irrespective of their instance?
YesCan I browse Lemmy if my instance is overloaded?
You can browse, but you can only vote, post, and comment from your home instance.If not, can I seamlessly move to a different instance?
Not as far as I know, but I’m very new to the fediverse as well. Your account is tied to your instance, but there is nothing preventing you from having accounts on multiple instances. You can even choose the same username! Usernames are @usernameFor example, I’m @Technoguyfication, but I also have a mastodon account @Technoguyfication. I can browse Lemmy/Kbin/Mastodon/etc. from any of my accounts, but anything I post or comment will be from the account I’m using at the time.
This was helpful. Thank you
Wait… So you can actually browse Lemmy from your Mastodon account?! So you can login?
No. You can’t login, but you can see posts from Mastodon. For example, when I made my previous comment, I got a notification on Mastodon that I was mentioned in this thread by my kbin account.
https://toast.ooo is accepting registrations 🎉
It seems like a common issue among ActivityPub services that people flock to the most popular instance and this causes problems. Why can’t load balancing happen transparently? It seems like the main thing that actually makes a difference between which instance users want to join is what the moderation will be like. Like I don’t want to be forced to sign up for an instance with a high amount of censorship compared to the rest of instances.
So maybe user registration should start from a centralized site that can describe the trade-offs of joining the various instances, and users don’t get to select their specific instance by default, but rather they select based on a loose moderation policy, and then load-balancing occurs on the backend.
EDIT: I also want to be able to migrate between instances without losing my community subscriptions.
This is inevitable if feddit is going to become mainstream. People have a herd mentality, if Lemmy is going to become popular there will always be a handful of instances that are much more popular than the others. These popular instances will need to scale (both vertically and horizontally) while the smaller instances will probably keep getting by with a single server. This is the same way email providers work, half the people I know use gmail, and most of the others use another large provider like yahoo or hotmail. It’s just the way this is going to have to work. People want to join an instance with their friends, even if they’re all federated together. They want to know that the instance they sign up for has peer approval and it’s already a tried and trusted one.
Honestly the whole fediverse feels too fragmented for the average user, myself included. I chose lemmy.ml because I want to be where everyone else is. I’m not sure I entirely understand how this all works, but therein lies the problem for the average user.
Sadly, I feel like the Fediverse, based on ActivityPub, was fundamentally designed wrong for scaling potential. I do like Fedi and I like ActivityPub, but I think instances should not have to be responsible for all of this:
- Owning user accounts
- Exclusively host communities
- Serving local and remote users webpages and media
- Never going down, as this results in users and content becoming unavailable
Because servers “own” the user accounts and communities it’s not trivial for users to switch to a different instance, and as instances scale their costs go up slightly exponentially.
I wish the Fediverse from the beginning was a truly distributed content replication platform, usenet-style or Matrix-style, and every instance would add additional capacity to the network instead of hosting specific communities or users.
I guess it’s a bit too late for a redesign now… Perhaps decentralized identifiers will take us there in some form in the future.
Not sure why you reference Matrix, which has even worse scaling issues as it indeed tries to replicate nearly the entire network on every server.
The Fediverse is really just working how the general web does, just with some standardized API for websites to interact. It’s not perfect, but it works and has proven to be relatively scalable.
It sounds a bit like you had a bit too much of the Bluesky cool-aid, which indeed replicates nearly all of the mistakes of Matrix and makes it impossible to scale via small community owned servers instead of big company owned data-centers (which might be by design?).
Well yeah, point taken that replicating everything everywhere and forever might be impossible. But I do believe at a minimum my identity should be portable and accessing Fedi (ie. in microblogging: posting and viewing a feed of the latest posts of my follows) should be decoupled from which instance I pick to access the Fediverse.
I don’t particularly like how owners of instances which grew are now essentially locked in to having to spend 100s or 1000s of dollars a month keeping their now expensive instances running and providing service. This is a bad place to be for a platform ran by volunteers. Letting instance owners scale their service down as well as up would be ideal. But this requires at least decentralized identity, and at best some form of content hosting redundancy…
It’s easy to say the current architecture of Fedi works when it’s still small. Your instance has 139 users… That’s not intended as a slight. Hosting instances is good and I applaud you for it! But I wish it were easier to more equally share the load once the platform becomes more popular.
Is there any group of devs that work on this issue that you know of? I’d be interested in looking into it.
No. And I think it’s a really hard problem. poVoq was right to call me out on full replication being a bad move, because duplicating all content on every server is obviously inefficient. But a solution in-between, with decentralization and redundancy, is probably a very complex challenge. Doesn’t seem impossible, but very complex network protocols rarely seem to succeed.
Edit: Sorry I was still thinking about some fabled perfect protocol. But if you’re looking into decentralized identifiers, W3 is working on one approach. It’s not something I have seen used anywhere or integrated with ActivityPub yet, but that could be the future I’m hoping for. Probably.
It almost sounds like you’re describing RAID 5 of content across fediverse servers.
Something like that. But also with fully decentralized identity. So all content is signed by a keypair which is local to the user, and can be used to access Fedi through arbitrary instances. Probably I am too wishful.
I could envision a 2nd class of server, running something like OAuth/OIDC, which handles the authentication into any Lemmy instance (or better yet, any ActivityPub based instance).
This server would also be self-hostable, and provide only authentication services, so it would be rather lightweight. But would help reduce the load on the content servers.
While it might not be too late for that update, it would require some reconciliation to happen. There’s the potential for multiple users and communities of the same name across servers that would need to be considered.
Yeah, the fact that the user auth and permission models were intentionally left out of the W3C spec initially really ended up locking ActivityPub into particular dialects and patterns that are now proving problematic for scaling.
I’m not sure it’s 100% too late for a redesign, though. The committee is still active and the Fediverse could still theoretically grow by an order of magnitude or two. Does that seem likely right this minute? No, but sometimes that kind of vision is what an ecosystem needs.
I’ve made https://lemmy.antemeridiem.xyz/ to help take off some of that load. New registrations are welcomed and it should be maintained for a very very long time 🎂
First post for me!
Sorry, I applied and got approved here. Still waiting to hear back from beehaw…
I’m really digging this UI compared to Reddit, but I am 99.9% a mobile user via the native Reddit app (don’t @ me!)
I am very tempted to setup my own instance. Wondering what resource usage looks like for an instance.
Yeah, I think I have two accounts (I registrated in a community and then came here and had to create another one because I couldn’t log in). It’s kind of confusing for people who are not as tech savy as myself.
Well, my understanding is your user exists on whatever instance you signed up on. You could technically create users on every single instance, but that is not necessary. You only need one user to exist somewhere, and then you can subscribe to, and post to communities on other instances.
For example: from lemmy.ml, if you search for
!gaming@beehaw.orgyou can then open the sidebar and subscribe to, and post to, the gaming community on beehaw.org with your lemmy.ml user.!gaming@beehaw.org is not the same community as !gaming@lemmy.ml
For example: from lemmy.ml, if you search for !gaming@beehaw.org you can then open the sidebar and subscribe to, and post to, the gaming community on beehaw.org with your lemmy.ml user.
This was really helpful, thank you!
An easy way to understand this is that instances are like email providers. You can sign up on Gmail, but still email someone using Outlook or something else.
That analogy makes a lot of sense. Very helpful to new users
@nutomic@lemmy.ml what kind of hosting do you guys use for lemmy.ml? At the time of writing it looks like you have around 33k users and around 2k active. What does that look like for resources consumed?
New user,how do I donate / tip to help you peeps cover server costs? It wasn’t directly obvious how to do it; apologize if it’s a big button right on a page that I missed.
Here. Liberapay is preferred.
Done my bit, started with a 5 dollar donation, monthly!
For future reference, the heart button up top is for funding.
