There’s a fair bit computers need to do to get all the pieces of their operating system up & running! This page will explore several components invoved, & the commands which can be used to configure it.
Shadow
We generally want to prevent total strangers from using our personal computers, so Shadow provides libraries & commands for checking whether you know some shared secret. This is an enhancement upon earlier UNIX authentication which was trivial to bypass due to it’s access control not being properly considered.
LibShadow
Much of the logic for managing the accounts database (Linux itself only understands user-IDs not user-accounts for authorization) is in a shared library between all the Shadow commands. And presumably other libraries can use it too!
The library has accessor for global program-name & log-FD variables.
There’s wrappers around the “utmp” file, with a global filedescriptor.
There’s utils for duplicating or freeing entries.
It implements it’s own exec-in-new-fork routine. This may be used to run a “sss_cache” command to clear secret’s SSSD cache, this command will be described later.
Or there’s a exec-in-new-fork-&-wait routines, refered to as “parts”.
Duplicate & free routines for a different kind of entry, with it’s own database read into thread-safe memory.
There’s a fair few routines for manage the Shadow TCB file, including creating it with appropriate access control before extremely carefully moving it into place, & for removing it. Has a global for the TCB user.
There’s a “common I/O” support utility powering all these in-memory databases, managing doubly-linkedlists reflecting any changes in the file from which it was parsed. It may callback to it’s wrappers via a methodtable. Includes sorting via an array & careful file opening.
There’s a parser for lines of the password-entry file.
There’s a routine for running the nscd command, which again I’ll cover later.
There’s wrapper around mutex-locking routines.
There’s routines wrapping /etc/nsswitch.conf parsed to an in-memory list under mutexes.
There’s wrappers around SELinux routines integrating into LibShadow’s logging & cleanup.
There’s a parser for group entries.
There’s a wrapper around unsigned-long or long-long or plain-long parsing.
There’s carefully error-handled fgets & fputs (actually looped fputc) wrappers.
There’s a routine to prompt for new field values & normalizing it with unchanged default. Alongside a validator.
There’s a parser for shadow password entries.
There’s more SELinux wrappers, carefully error-reported.
Building upon these simpler routines, as well as it’s own variation of libmisc, & a validated crypt wrapper; LibShadow includes a handful of more complex systems.
There’s routines for checking if a user can log in at the current time, checking against an array of allowed times after linear-scanning the users table & parsing /etc/porttime.
There’s parsing of the groups file.
There’s linear scanning of the shadow file off disk/SSD.
There’s an in-memory groups database parsed from the /etc/groups file. Like the others.
There’s a parser & parallel-mapping for a configuration file, which has an optional main() function validating the config.
A “common I/O” sublibrary is lightly wrapped to manage a mutex-protected list parsed from the shadow file. Likewise there’s various wrappers around the shadow database, with a methodtable.
The authentication routine validates the given “reason” is appropriate for it to handle, that it’s running as root, & a cypher is given. It might then on supporting systems call skeychallenge, outputs the configurable (with localized default) login prompt possibly preceded by that challenge & reads the password without echoing it on stdout, the password is “encrypted” (“hashed” is probably the proper term…) & compared against the given “cipher” (stored password?). If S/Key is supported & no password is given it prompts for some no-echoed response & calls skeyverify. The password is carefully erased.
And finally there’s a submodule wrapping that Common I/O system to manage subordinate user & group IDs, implemented much like a memory allocator with linear-scanning. This has quite an extensive public API.
LibSubID
There’s barely anything to study in LibSubID. It’s a shallow wrapper around the subordinate user/group IDs whose allocator I mentioned yesterday as part of LibShadow.
Includes an initializer wrapper the progname & logfd global setters.
Shadow Commands
LibShadow includes several commandline tools you might be familiar with. Going roughly from most trivial to least…
getsubids wraps subid_get_gid_ranges or subid_get_uid_ranges serializing the output. Lacks normal internationalization (since there’s barely any text to localize) & flags-parsing code.
get_subid_owners works similarly. As does free_subid_ranges.
check_subid_ranges is even more trivial wrapping have_sub_uids & have_sub_gids.
nologin syslogs these attempted logins & indicates failure.
new_subid_range wraps subid_grant_gid_range or subid_grant_uid_range.
After configuring LibShadow, envvars, signal handlers, internationalization, & logging expiry parses a couple commandline flags, looks up the password entry, wrapping either agecheck+isexpired or expire.
After initializing Sysconf, internationalization, & logging if called with no arguments groups retrieves & iterates over the groups.
First these iterations checks if the “primary group” is in the list. If so it outputs resulst of getgrgid. Regardless the main iteration calls getgrgid for all groups outputting each’s results.
If args were given it iterates over the groups (after checking getpwnam) outputting the names which were on that list. After which it might output the results of a special getgrgid call and/or an extra newline.
After initializing internationalization, some vars & SysConf id validates the count of commandline args (possibly ouputting usage), retrieves the process’s user & group IDs followed by the corresponding password/group entries outputting them if non-NULL, possibly iterates over the user’s groups looking up & outputting each of them, & cleans up.
After initializing LibShadow & validating at least 2 commandline args are given newuidmap looks up the PID for the first argument, opens its /proc/ directory, finds the user’s password entry, performs validations, parses remaining commandline args as ranges, validates each of those ranges, & outputs them to the proc dir’s uid_map file.
newgidmap works very similarly!
Unless the command’s disabled, after initializing LibShadow & internationalization grpunconv has a special -R case.
Then it parses the remaining -h commandline flag, checks that /etc/gshadow is present & can be opened under a lock, & linear-scans that file to find the entry to overwrite setting the password.
After init’ing i18n logout might fork if the debug flag is set, initializes logging & LibShadow, & repeatedly iterates over the utmp[x] file checking if they’re allowed to be logged in at this time. If not it forks, sends a message to their terminal, killing their process, & syslogging.
Amongst them I see routines for parsing the superusers group to determine whether the user is authorized to use su or sulogin.
After init’ing the terminal, LibShadow, i18n, & envvars sulogin considers overwriting the standard stdin/etc, validates the current env, saves envvars, configures a timeout, iterates over the passwords file to get root’s password comparing against userinput, carefully cleans up, & runs the given command.
After initializing LibShadow, internationalization, & logging pwunconv parses a couple commandline flags (-R handled seperately), validates the environment & opens the passwords file under lock, iterates over that passwords file & corresponding shadow passwords, considers calling pw_update for each, & cleans up with error reporting to both stdout & syslog.
pwconv has similar initialization & cleanup, but with 2 iterations over the passwords file.
The 1st removes shadow passwords for non-existant accounts. The 2nd moves passwords out of the general access passwords file to the limited access shadow file, replacing those passwords with the “x” indicator.
There’s a shared utility consults /etc/login.access syslogging invalid syntax which controls who can login as who given password authentication.
After initializing LibShadow, internationalization, & logging vipw parses a few commandline flags validate no args remain.
Depending on flags vipw/vigr may:
- Given -p & not -g (main differentiator between these commands) & -s it might lookup the TCB entry & will run the core logic under shadow lock; followed by a warning message.
- Given -p & not -g & not -s it runs the core logic under password lock, then checks whether to warn about the need for the -s flag.
- Given -g or not -p & -s with a buildflag set it runs the core logic under SGR lock followed by a warning to use
vigr.
Otherwise it runs the core logic under the GR lock, possibly followed by a warning (if SGR file is present) to provide the -s flag. Regardless it flushes several caches.
This core logic involves possibly generating a backup TCB file, checking the file to edit exists, (re)configuring SELinux on it, claiming relevant privileges & locks, opening the file, creating a backup, looking up the configured editor, fork/exec’ing that subcommand in foreground, & once it closes extensively cleans up.
After initializing LibShadow, internationalization, & auditting lastlog parses/validates commandline flags ensuring no args remain, opens /var/log/lastlog retrieving its filesize, either updates or prints to it, & closes it. Updating involves between validation & flushing querying password entries to update. For each relevant entry it seeks to that offset in the lastlog & writes a binary structure with audit logging. Printing gathers more info.
After initializing LibShadow, internationalization, & logging grpconv parses a couple (-R handled specially) commandline flags, opens the group & shadow group files under lock iterates over the shadow group file removing groups not in the group file, iterates the groups file to move over passwords, & cleans up.
After init’ing LibShadow, internationalization, & logging pwck parses a few commandline flags (handling -R specially), opens the password & shadow password files under lock, either validation or sorts the files, & cleans up with error messages & exit codes. The validation involves iterating over each entry skipping NIS lines, & prompting what to do about any syntax errors, duplications, etc it finds depending on their severity. This is implemented seperately for the shadow file.
After init’ing LibShadow, santization, i18n, & logging chfn parses commandline flags (handling -R specially), retrieves name password entry or the user’s own, validates it’s not a NIS account & the users’ permissions, copies over old fields, prompts for new fields they’re permitted to change if none of those values are given in the commandline flags, validates the new fields (I don’t like the ASCII restriction I see…), writes this data to the end of the GECOS file & to the appropriate location in the password file, syslogs, & cleans up.
After initializing auditting, internationalization, LibShadow, logging, & the envvars, newgrp retrieves the user’s password entry, manually parses a - or -l flag, manually parses -c or retrieves the group entry, retrieves the grouplist repeatedly reallocating it’s memory & reporting errors, retrieves the specified group by name with error reporting, possibly checks whether we’re a member of that group & if not rereads the group, retrieves shadow group, checks permissions, considers syslogging, calls setgroups if available, closes files it’s querying, sets the process’s group & user IDs, runs the given subcommand if any, otherwise retrieves $SHELL envvar, possibly home directory & certain envvars, audit-logs, runs that shell, & reports any failures.
After initializing LibShadow, internationalization, & logging whilst sanitizing the envvars chage retrieves the process’s user/group IDs validating permissions whilst parsing/validating commandline flags ensuring a single arg remains, validates the shadow-password file exits, opens password & shadow files under lock, drops privileges, finds password & shadow (& possibly TCB) entry for given account, considers outputting expiration times if requested & permitted, prompts for new values if not given in flags, audit logs which fields will change, updates the shadow password entry, cleans up & syslogs.
After initializing LIbShadow, internationalization, & logging whilst parsing commandling flags (handling -R special) newusers checks privs & that the shadow files are present, opens all the accounts files, repeatedly reads lines from stdin, reports any errors, cleans up, & copies info over to PAM if supported. For each stdin line once validated newusers splits it into 7 fields, locates the password entry, validates a bit more, writes out the parsed group then user if valid, locates the password entry, generates & writes the hashed password, outputs password entry, & outputs/allocates subordinate IDs.
After initializing LibShadow, internationalization, logging, & exit handlers whilst parsing a few commandline flags (handling -R & -P specially) validating that 1 arg remain groupdel may (if ACCT_TOOLS_SETUID & USE_PAM are set attempts to authenticate via PAM logging any errors. If the SHADOWGRP` buildflag is set it validates that file’s presence. It looks up the group ID for given arg reporting errors.
If USE_NIS buildflag is set group_del validates this isn’t a NIS group reporting errors. Regardless it might iterate over the passwords file to validate we’re not deleting any user’s primary group. Regardless it opens all the group files registering cleanup callbacks, wraps gr_remove & possibly sgr_remove, & cleans up flushing caches.
After init’ing LibShadow, i18n, & logging whilst parsing commandline flags (handling -R specially) & checking if shadowfile presence groupmems …
groupmems validates/normalizes selected group fallingback to the current user’s. Privileges are validated, with PAM if supported. Groupfiles are opened & the given group located. Before cleaning up it chooses a codepath of either:
- Iterating over parsed members outputting them.
- Adding the group from parsed list in both files if not already present, saving results.
- Removing a user from the parsed list in both files if present, saving changes.
- Nulling out those lists, saving changes.
After initializing LibShadow, internationalization, logging, & exit handlers whilst parsing/validating ensuring single arg remains commandline flags (handling -R & -P specially) groupadd meanwhile validates privileges with PAM if supported, validates shadow group file’s presence if it’s buildflag is set, opens group files under lock registering cleanup functions, locates the group ID, writes a new entry if not already present, & cleans up flushing caches.
After init’ing LibShadow, i18n, logging, & exit handlers whilst parsing flags (handling -P & -R special) validating single arg remains groupmod checks with PAM if supported, validates shadow-group file’s presence if it’s buildflag’s set, validates it’s not a NIS group if that buildflag’s set, checks if it already exists, & that the name is valid. Then it locks the group files, reports changes to audit log, opens those files, locates & overwrites the group, & cleans up flushing caches.
After santizing envvars then init’ing LibShadow, i18n, & logging whilst parsing a couple flags (handling -R special) ensuring at most 1 arg remain & checking whether we’re root chsh checks whether the first arg corresponds to a valid user looks up the specified user falling back to our own, validates this is a local user, checks privs (including with PAM if supported), fills in -s fallback with user prompt, validates, overwrites password entry, syslogs, & cleans up flushing caches.
After initializing logging, internationalization, LibShadow, & std I/O buffering whilst sanitizing envvars gpasswd checks shadow-group file’s presence if it’s buildflag’s set, looks up your password entry, registers exit handler, parses/validates commandline flags ensuring single arg remains, duplicates the entry, checks privs, mutates the group depending on flags (disabling passwords via magic strings), configures signal handlers, prompts for password validating you can retype it, hashes & sets that password zeroing out it’s RAM for security, switches to root reporting any errors including to syslog, initializes password database, opens relevant files, overwrites the group files, & cleans up flushing caches whilst logging successes & failures.
After initializing LibShadow, internationalization chgpasswd parsing commandline flags (handling -R special), opens log, checks privs with PAM if suppored, checks shadow-group file’s presence if it’s buildflag’s set, opens relevant files under lock reporting errors, & reads each line of stdin before reporting errors & cleaning up.
For each valid stdin line chgpassword parses it, checks the configured hash function & hashes the password with it, locates the group (in both group files, depending on buildflags), sets the new password, & writes updates.
chpasswd meanwhile works similarly except it adds a configurable salt, & has extra per-item PAM authentication if supported.
After initializing LibShadow & i18n faillog parses/validates commandline flags (handling -R special) validating no args remain, opens the faillog file retrieving its size & reporting errors, possibly reads/updates locktime within that file, possibly does same for max-fails, possibly resets fail count, otherwise prints relevant entries, & cleans up reporting errors. These changes can apply to a single user-ID or a range of them, giving each support function a wrapper handling this branching.
After initializing LibShadow & i18n grpck parses commandline flags (handling -R special) whilst opening logging. Then accepting 2 additional args grpck opens the group files under lock reporting any errors including to syslog, either sorts or validates them, closes the files, cleans up flushing caches, & reports any errors including in the exitcode.
Validation involves iterating over the group file skipping over NIS entries reporting any syntax errors (which it prompts to delete), duplication, fields validity, & correspondance in Shadow file. Has seperate validation for shadowfile.
After sanitizing envvars then initializing LibShadow & internationalization passwd parses/validates commandline args (handling -R special) whilst opening logging & checking whether we’re root, retrieve our own entry in the passwords file, checks the next arg for the account to set fallingback to our own, verify no further args are given, handle -a flag validating other flags aren’t set by iterating the passwords file & outputting each entry, validates flags further, validates the user’s presence, checks privs with PAM if supported, validates we’re root or changing our own password, given -S exits outputting our entry, either calls PAM routines or having re-authenticated the user prompts them for their password twice validating against LibCrack, cleans up flushing caches, & reports results.
After initializing i18n su initializes LibShadow to save info from our password entry, opens logging, parses commandline flags & remaining args, resets envvars, checks privs with PAM if supported, looks up our password entry for more privs checking & reauthorization, fills in fallback values, logs this action, configures new credentials with or without PAM, writes to audit log, reconfigures envvars again, reconfigure shell, cleans up, & runs the shell command.
After initializing LibShadow & internationalization userdel parses a few commandline args (handling -R & -P special) validating a single arg remains whilst opening logging, authorizes the action with PAM if supported, validates relevant files are present, runs a configurable pre-action script, reads specified password entry, takes into account the given prefix, configures TCB user if supported, validates we’re editting a local user who isn’t actively logged-in, opens relevant files under lock auditting failures, wraps pw_remove & spw_remove, wraps sub_uid_remove & `sub_gid_remove if supported, audit & sys logs the deletion, iterates over groups twice to delete the user from each syslogging each time, deletes their mail directories if -r given, deletes, validates their home directory to decide whether to follow through with deleting that, deletes SELinux user, cancels crontabs for the user, & cleans up flushing caches.
After initializing LibShadow & internationalization usermod parses -R & -P flags, opens logs, consults sysconf, cehcks relevant files are present, parses & validates remaining flags ensuring a single arg remains, checks that user isn’t logged in, authorizes with PAM if supported, checks TCB cache, opens relevant files, modifies appropriate entry as specified, adds & removes from specified group entries syslogging any errors, adds & removes sub user & group IDs, cleans up flushing caches, reconfigures SELinux if supported, moves the home directory if its path was altered, moves the mailbox if its path was altered, updates last & fail logs, & chowns the user’s home directory if needed.
usermod works much like many of the others I described. To update the userfiles it locates the entry to edit cloning it, handles passwords specially, & wraps [s]pw_update & [s]pw_remove. Group updates as per usual adds removes from appropriate lists. And it can handle reallocating sub-IDs.
Similar for useradd, expect it runs configurable pre & post scripts, handles a special defaults file, …
ensures the given user & possibly group doesn’t already exist, allocates user & group IDs, resets logs as it creates the users, possibly adds a new corresponding group, & when done it inserts an entry into the tally log, might create the home directory possibly copying over a template directory, & possibly create a mail folder. With correct permissions. Before running a configurable post-action script.
After sanitizing & initializing envvars whilst initializing internationalization & LibShadow login checks whether we’re root, parses/validates commandline flags & remaining args, quits if we’re not in an interactive terminal, retrieves utmp dir, performs stringent authorization, retrieves terminal name, configures $REMOTEHOST envvar, tweaks parsed flags, opens logging, configures umask if PAM unsupported, handle -p, configure $TERM, copies over some hardcoded envvars followed by those given in commandline args, prepares some text for logging choosing a source identifier, looks up & configures login timeout preparing error message, looking additional authorization configuration envvars, performs the authorization in a loop with or without using PAM, cleans up, finalizes envvars, performs various logging, sets the new user-ID, reinitializes internationalization, handles whether or not the user’s “hushed” possibly via PAM, resets signal handlers, does more syslogging, closes the log, & execs the login shell.
Each iteration of the no-PAM authentication loop involves cleaning up, prompting for the username if needed, retrieves where to log failures, looks up the user & checks their password isn’t locked & whether a password is even needed & whether the password’s in the shadowfile, wraps pw_auth, syslog failures & flags failures appropriately, if there were a failures it carefully considers whether to reauthenticate, pauses for configured amount of time, & outputs an error message.