Questions

• FS: FHS
• systemd-nspawn, systemctl
• WM, SM, compositors, etc
• Daily best-practices, basics of working with servers

Intro

Main Linux components include:

• Boot system
• Kernel
• Init system
• Package manager
• Graphical interface

Boot system

Grub or systemd-boot, loads kernel and starts init.

Kernel

Interacts with the hardware, some low-level processes (e.g. wireguard).

Init system

Primarily systemd (alternatives include runit, etc). Starts long-running tasks.

Package manager

System responsible for securily installing applications into the environemnt.

Graphical interface

Session managers, window managers, graphical applications, etc.

FHS

Official documentation.

Common locations

• /bin - user binaries.
• /sbin - system binaries.
• /lib - .so files, internal binaries, kernel modules.
• /etc - configuration files.
• /tmp - temporary files, cleared on boot.
• /var - runtime files.
• /opt - locally installed applications.
• /boot - kernels, .efi and such.
• /dev - pseudo. all peripherals and block devices.
• /run - normally tmpfs, used for runtime data like secrets that shouldn't be written to disk.
• /proc - pesudo. process tree.

Systemd

Units

Describes various components operated on by systemd. Uses

systemdctl -t help to see all types, systemctl list-units to see all units available on the machine.

Common types:

• service: executable tasks.
• mount: fstab but systemd.
• target: group of services.
• timer: timetable to run services.
• slice: group of services belonging to a single user.

Common targets:

• poweroff.target: shutdown.
• rescue.target: rescue shell.
• multi-user.taget: nongraphical multiuser.
• graphical.target: graphical multiuser.
• reboot.target: reboot.

Unit example:

[Unit]
ConditionSecurity=audit
Description=Kernel Auditing
[Service]
ExecStart=@audit-disable audit-start
ExecStop=@audit-stop audit-stop
RemainAfterExit=true
Type=oneshot
[Install]
WantedBy=basic.target

systemd-nswpawn

Gives you some interesting controls, integrates with the systemd ecosystem (targets, timers, etc.). Otherwise not much better than any other container runtime.

Logging

All systemd services log stdout/stderr (configurable) to joutnalctl, which also includes logs from systemd itself.

Utils

systemd-run can launch a binary in an environment controlled by systemd.

Graphical environment

Xorg and Wayland are methods of displaying objects to screen. Xorg is a single server and wayland is a protocol, which requires any wayland server. Session managers handle login, run specific window managers from a specific user. Window managers handle displaying windows. Graphical environment (Gnome, KDE, etc.) do all of that.

Xorg and Wayland

• Xorg is older and generally more stable.
• Wayland is newer, more lightweight and less stable.

When building graphical environemnt you should choose between Xorg and Wayland, unless you use a GE which handles that for you.

Session managers

Just use sddm.

Window managers

Can be tiling, do window placement, run on Xorg or Wayland.

Tiling on Xorg:

• i3 and forks
• bspmw
• Xmonad

Tiling on Wayland:

• sway and forks
• hyperland
• awesomewm

Terminals

See terminals for tty internals.

Main function of a terminal is translating keystrokes into tty actions and displaying its output.

Can run on Wayland or Xorg, provide various features depending on how much you want. Important features include tabs, GPU rendering, ligatures.

Examples:

• kitty - also supports images.
• alacritty
• st

Desktop environments

KDE, Gnome and Xfce are primary examples. Run on Wayland, but that's configurable. Chose whichever looks better to you.

Other components

• Sound: handled by sound servers like pulseaudio and pipewire.
• Pop-up notifications: handled by a notification deamon like dunst.
• Visual flare: includes transparent windows, blur, etc. On Xorg rendered by compositors like picom.
• Bars: self-explanatory.

In graphical environments main avenue of IPC is D-Bus. It allows applications to trigger code execution in other applications, get structured information from them. Example of D-Bus usage is reading currently playing song from a web-browser. KDE, GNOME and other DE's utilize it heavily for communication between different components or for providing interface to lower-level components like power management.

Servers

Servers work generally the same apart from not having graphical environment. Newer approach to servers sees them as fully temporary - services should be independent of their environment, servers themselves should be declarative as much as possible. Popularized by cloud computing, this approach makes for simpler deployment, scaling, etc.

Declarative approach

Under a declarative approach deployment of a service doesn't require any actions apart from realising saved configuration. Examples of fully declarative tools include Terraform and NixOS.

Impermanence

A concept, meaning that almost the entire FS tree is treated as temporary files. Under this concept, any necessary state is declared ahead of time and stored separately. Containers are impermanent, and this concept can be applied to physical servers too.

Best practices

• Only use package managers for installing software. If something isn't available then package it.

Tooling

Key tools in running a Linux installation include:

• shell
• find/fd
• grep/ripgrep
• sed

Consider vim and tmux.

Backups

If storage allows, use backup software like btrbk, restic or burp. Encrypt backups if needed.