"Operating Debian without systemd is a pleasure and every time one of my friends has some systemd-induced lossage I get to feel smug."
How common is a systemd-induced loss? I've never had one, and I've never seen or heard of any. Is this a larger problem that I've somehow missed? (This is an honest question, I was just surprised to read this)
I've never had data loss (assuming that's what "lossage" means), but here's an incomplete list of problems I've experienced on systems running systemd:
* System randomly taking 5+ minutes to boot
* Daemons sometimes not starting on boot
* Logging sometimes just stops (I confirmed this was caused by a systemd bug)
* Logging in over SSH starts taking 2 minutes and can only be fixed by reboot (I confirmed this was caused by a systemd bug)
It's not that sysvinit scripts are without bugs, especially race conditions and daemons inheriting environment when you invoke a restart script, but systemd appears to be distinct in that a seemingly-quiescent system can just suddenly stop working correctly. They had a low bar to clear to improve on the reliability of sysvinit systems, and yet they failed to clear it. That's not surprising given its excessively complex design.
"systemd-analyze blame" can help identify issues relating to boot time
* Daemons sometimes not starting on boot
This most often occurs for me when services don't start quickly enough (too much contention on spinning rust and aggressive default timeout settings) so I have to adjust startup timeouts. "systemctl --state=failed" along with "systemctl status X" can usually help narrow down why your services failed to start.
* Logging sometimes just stops (I confirmed this was caused by a systemd bug)
I've found journald can get forcibly restarted if there is a lot of contention for disk iops and it can't checkpoint. I frequently run into this issue on a server that is used for backups for about a dozen or so clients that relies on filesystem snapshots.
* Logging in over SSH starts taking 2 minutes and can only be fixed by reboot (I confirmed this was caused by a systemd bug)
I've sometimes had this happen and for me it seems to be related to systemd-resolved deciding to stop responding to queries and the only solution I've found so far that actually works is to restart the system.
Despite the bugs I still prefer systemd over sysv-init - being able to run something like "systemctl status" and "systemctl --state=failed" to inspect the state of all services on a system has been invaluable to me.
> I've sometimes had this happen and for me it seems to be related to systemd-resolved deciding to stop responding to queries and the only solution I've found so far that actually works is to restart the system.
I hit this all the time on my Arch laptop and was a major impetus to switch to Void.
I really wish I could reproduce the thing on demand. I'd file a bug report but it has happened only a handful of times to me in the past 2 years and only on a single server. :-/
Heh I absolutely love Arch, and probably would never have made it to the point of being able to do Void or FreeBSD installs if it wasn't for Arch and its excellent docs. But yeah, some kinda annoying stuff with it lately.
Hackers used to say "win" to mean succeeding at some technical task, particularly if you gain some desired advantage, and "lose" to mean failure or being put at a disadvantage. Windows 11 is a losing operating system. Tree-sitter is a huge win for code editors. Etc.
"Lossage" means "the consequences of a lose" even if no data loss occurs. Some people consider systemd with its complexity and tightly coupled components to be a lose.
I'm still livid at this Luca guy gaslighting people on the "correct" behavior of a feature people were using for years already. They re-purposed an existing sleep mode and changed it entirely to mean something else. Originally you would set a timeout that your laptop would suspend and then hibernate. They changed it to mean that your laptop suspends until your battery hits 5% and then hibernates. As if people want to open their laptop the next morning only to have no battery left.
Ironically, the stated reason for this change was to "prevent data loss". Guess what they did? They introduced a bug in this "fix" that guaranteed your laptop would not wake from hibernate. You had to reboot the damn laptop each time you woke from hibernate because it would get stuck on a black screen, thus "losing" your data. I "lost" more data from their stupid fix than anything else.
They admit the original feature was broken because it didn't take into consideration for battery percent. Fine. But there was absolutely no reason to change the behavior. You can do both! The only reason I can think of is you just like being an incredible asshole to people. Which is obvious from that thread. Do they even use a laptop? Why would I want a dead battery next time I open the lid???
> I'm still livid at this Luca guy gaslighting people on the "correct" behavior of a feature people were using for years already.
As a fellow (though not lately very active there) systemd maintainer, reading his comments in that issue is a real bummer.
I'm not sure I'd call it "gaslighting", I don't think he was even bothering to grok what people were asking for. Not until it became a pile-on of additional systemd members @yuwata and @DaanDeMeyer.
Maybe he's checked out a bit with all the drama going on over at BlueHat?
I’ve never heard of one either. While systemd is certainly more complex than old school init scripts, and arguably deviates quite a bit from the “Unix philosophy,” it seems some people just have an irrational hatred towards it. To the point of making up problems with it that I’ve never heard anyone actually having.
There are a few reasons why systemd left a bad taste in the mouths of many Linux enthusiasts:
Systemd replaces (or tries to replace) whole swaths of OS functionality all at once, instead of addressing each piece individually. As a result, all of the systemd pieces are tightly integrated and nigh-inseparable, and critics claim this was intentional so that distributions could not easily pick and choose the parts they wanted. My understanding is that this is better now. In my experience, some pieces are better than others. (Systemd-timesyncd is absolute rubbish compared to chrony, for example.)
Although the goal of writing a better init system is laudable, a lot of people who looked at it early on not impressed with the implementation (a lot of NIH wheel-reinventing) and mediocre code quality.
It was introduced into Fedora rather suddenly with seemingly little warning, community discussion, or press coverage. In the Linux world where big things change slowly if at all, other popular distributions seemed to adopt it fairly quickly, one after the other. This raised a lot of conspiracy-like speculation about how that could have possibly happened.
The developers are famously obstinate about their technical decisions and often argue in, close, or ignore bugs asking them to consider a different direction for specific low-level issues. There have been a few public disagreements over "correct behavior" when integrating with other projects (e.g. the Linux kernel).
Had a problem with systemd-resolved yet again the other day. Blatted resolv.conf and job done, but between systemd and related things (network manager etc) it just breaks the way things have worked for decades and gives many people no benefit.
This isn’t all systemd of course, it’s the distros that choose to use it.
Two systemd things that I disable straight away on every box: systemd-resolved, and systemd-timesyncd. The former tries to be "smart" and therefore tends to mess up DNS in very simple scenarios (such as a box in a datacenter with a static IP) and the latter is just plain rubbish compared to chrony.
It's been a blessing for me. It's so easy to see the process hierarchy, easy to create units and timers and dependencies, and with services being combined into cgroups the concept of a zombie process was basically eliminated.
I guess I get that init was simple, but it didn't care about the process lifecycle. You always needed to have an incredibly smart init script or watchdog service to manage that.
I did have systemd induced problems but not recently. It was beta quality software when it was first introduced. Even on Debian which wasn't the first distro to ship it. Problems were mostly related to the different naming schemes they introduced to solve problems where the name of a disk or a network interface could randomly change and cause issues. Incidentally that's exactly the areas where it bit me. On a remote system the issues could have been very serious. I did not have such problems in at least 10 years. Ubuntu's new networking changes are causing a similar problem on the latest LTS upgrade. The package manager does not complain about anything nor migrate the configuration but systems end up with no network. These issues are a pain to resolve on remote bare metal servers, thus unacceptable. Yet we accept them. (Not really. There's already an exodus from Ubuntu, due to Snap)
I used Debian without systemd for a long time until it was no longer possible by upgrading. (I believe they make an effort to optionally run without systemd nowadays). In the meantime systemd got better. I still don't like it. It's like how people won't give btrfs another chance because it ate their data once 15 years ago. I think this is completely understandable. Thinking about the past, looking at the history, there's no reason to trust neither the technical skills or ethics of the people pushing systemd (or snaps) either.
Agreed, I'd say it's first on the system that isn't handling exiting/shutdown cleanly, and secondly on the distro config timeout value. I get this quite a bit on some machines (I use Fedora) and it is super annoying. I have yet to find which service is resonsible though as anytime I'm shutting the system down is inherently not a good time for debugging :-)
I have had one issue related to systemd; at one point when doing a routine update there was some dangling symlink issue that caused systemd to crash. This wasn't catastrophic; a lot of things didn't work, but I was able to cleanly save and exit out of applications, manually run a sync to make sure everything was flushed to disk, and then hard reboot.
systemd hasn't been without issue, but I've definitely had fewer issues with systemd than I did dealing with broken init scripts or upstart prior to moving to systemd.
I've never suffered data loss from systemd, but I certainly have encountered loss of functionality. There are a number of things that I just can't accomplish with systemd that I can accomplish without it.
Sure. My most recent headache is with using a USB wifi dongle. During system startup, I want to attach some remote drives over wifi, so I need to be able to do that after the wifi connection is established.
Systemd does not appear to be able to tell when that happens. It thinks the network is established before it actually is, and then the mount calls cause the startup process to pause for 3 minutes for those mount calls to time out before continuing.
The funny thing is that sounds like a perfect case for systemd: By eating so many parts of the OS (here, device management, filesystem mounting, networking, and service management), it should be able to create very explicit dependency graphs - "this mount depends on this network which depends on this device". I wonder where the problem is - can it not properly handle something between the device and network? Or is it hitting the "network is up" goal too early?
Thank you. I wish to add bluetooth dongle and expecting some headaches there. It seems setting up a full bluetooth stack would be challenging to say the least and I wander if systemd would contribute more to the challenge or to the solution.
While "lossage" in the context of an OS implies other things, in this statement it could mean time. I've had plenty of time wasted by systemd, as has pretty much everyone I know who has had to do more than basic tasks with systemd (and some people who were just trying to do basic things).
Very uncommon. I'm using it since it became default on Arch and never had any problems. Systemd is actually the best thing to happen to Linux. Luddites gonna luddit.
