Table of Contents
Sudo @ ICS
This page has been created to share philosophy and policy regarding ICS sudo and provide sudo alternatives. If you don't find the answer you are looking for here, or if you find an incorrect or incomplete answer, send mail to helpdesk@ics.uci.edu
Take-away
The big thing that everyone takes away from this document is that sudo root should be used in one of these three situations:
# To start or stop a service on a privileged port or via the systemctl command # Install packages via apt or yum. # To collect information or make configuration changes to the system that is not available to another user such as apache or postgres.
If your use of sudo usage doesn't comport to one of these three uses, continue reading or reach out to helpdesk@ics.uci.edu and we'll suggest convenient alternatives that your whole team can use.
Sudoers
Some ICS research hosts are cordoned into a security sandbox and users have access to sudo commands. See OS support levels
Philosophy
The system operations team strives to strike a balance between allowing researchers to self manage their computing resources and maintaining maximum stability, conformity, and security of the systems we manage. We acknowledge that it can take a little bit longer to reach out to helpdesk@ics.uci.edu, but it also allows us to provide updates safely, securely and uniformly across the entire ICS fleet. Our team makes best effort to avoid the role of the gatekeeper or to be a blocker. At the same time, we want to limit any critical problems, such as a broken package dependency, to derail your work when your team does have a deadline.
Delegation of Authority
With that in mind, we group sudo privileges into the following broad groupings:
- Senior team members have sudo access to update or make changes to the system.
- Other team members may have sudo access to commands that are ephemeral or informational.
We ask researchers for points of contact, typically one or two senior grad students that have familiarity with all the group projects. These team members will be called on by the systems operations group in the expanded capacities:
- coordinate system maintenance for the group
- sends/approves software change requests
- request visitor accounts
- control of group's slurm partitions
Common Pitfalls
Examples of ways that promiscuous sudo privilege can block research.
Update Alternatives Contention
Multiple versions of programs, such as Java or Python, are present on most linux distributions. On Ubuntu, switching the default java or python version is as simple as installing a package that runs the update-alternatives command. A change to the default version that isn't carefully considered and discussed ahead of time can disrupt projects and will be time-consuming to detect.
Broken Package Dependencies
The yum and apt package managers on occasion break and it can be difficult, if not impossible, to resolve nonsensical package dependencies. To reduce risk of problems we ask that package updates privileges be restricted to a small number of users.
Usage
Run the following command for a list of sudo commands available to you:
sudo -l
Filesystem (e.g. mkdir, mv, cp, etc)
Using the sudo command to create or edit files in this space is typically unnecessary and can be counterproductive for your team. Using root privilege escalation to create and edit files will block team members that do not have root privilege (ICS follows the principal of least privilege).
Alternatives:
Local Storage
All ICS systems have world or group writable local storage. Every system is going to have /tmp and /scratch directories. Many research directories will also have local drives mounted beneath /drv or /srv. Many other systems have undeployed storage. If you have questions about what is available on a particular workstation, please send mail to helpdesk@ics.uci.edu
Group accounts
If you need to share storage with your team, use your Groupleader Account (Linux). It works the same way as root sudo, but avoids the requirement that the entire team has root sudo. The group account can use all the same local storage as well as, on many systems, NFS storage.
Universal Student Access
All students have been given access to a limited number of commands on instructional hosts. Grad students will have the same privileges on research hosts. The commands are commonly used diagnostic commands that require root privilege to execute but are reasonably necessary for day to day work.
Getting More
If you feel that you require access to a command that is not on the list, please email helpdesk@ics.uci.edu.
In the cases that you have been granted sudo access to a host, please consider what it means to have that privilege.
- Broken or corrupt OS and system software will be repaired by total reinstallation.
- Backup of local storage is your responsibility. Mount your ICS Home Directory and copy important data there.
- Accounts are managed centrally by helpdesk@ics.uci.edu.
- Request permanent changes to the OS and system software from helpdesk@ics.uci.edu.
- Make no changes that would interfere with the ICS Computing Support group to manage the computer
- Make no changes that would prevent ICS Puppet from updating the configuration of the machine.
- The machines are monitored (grafanaand Icinga2@ics). Alerts will trigger a response.
Sudo Alternatives
Using Code in Openlab
We've seen a lot of students trying to install code. This isn't going to work because sudo, but also because the X Server isn't near fast enough.
Alternative 1: Use Jupyterhub@ICS: https://hub.ics.uci.edu
Alternative 2: Run VSCode locally but run programs on Openlab using “Visual Studio Code Remote - SSH”.
Alternative 3: Map your network drive and run code locally on your computer.
Package (yum/apt) installation
It would not be useful to allow everybody on the openlab to add or remove packages as they saw fit so we don't give out sudo permission for that. However, that doesn't mean that the package you want' isn't available or that you cannot install your own packages.
Modules
An alternative to update-alternatives
Natively, CentOS and Ubuntu may not provide the latest program versions. To address this, ICS Computing Support will provide locally compiled software as part of the ICS Software Library.
You may add these packages to your environment by invoking the module command.
Add a different javac to your path:
module load openjdk/11.0.2
Add slurm to your path:
module load slurm
Add julia/1.6.0 to your path:
module load julia/1.6.0
See which versions of gcc are available:
module avail gcc
Multiple Versions
An alternative to update-alternatives
On instruction and research systems, several versions of python are provided. These may be invoked by specifying the longer version: e.g. /usr/bin/python3.7
See what is available locally by using shell tab completion.
You can also see which versions of python are available by using wildcards:
ls /usr/bin/python* /usr/local/bin/python*
Python3 Packages
Most python and anaconda packages do not require root privilege to install. See the following URL for instructions on installing Python libraries, including how to upgrade pip:
DB Servers
The MySQL and PostgreSL servers run on unprivileged ports and it is best practice to run these as a non-root user. Please see this page for running MySQL as an unprivileged user. Please request a group account from helpdesk@ics.uci.edu if your team would like to share ownership of a MySQL server.
NodeJS
Compiling Software from Source
Software source can be compiled and installed to writable storage by any user without SUDO.
Use the –prefix option to rehome the package
./configure --prefix=$HOME/pkg/pkg_name/version
See this link for a little bit more information.
find
sudo privilege is not required to search areas of the filesystem that belong to you or public spaces. Due to our security policies many areas of the filesystem are not going to be accessible to root. Please reach out to helpdesk@ics.uci.edu if you feel you need to search an area of the file system that you do not have access to.
ldconfig
Set your LD_LIBRARY_PATH And LD_RUN_PATH instead. For example:
Add a lib directory in your home directory to be search when running programs (bash/zsh)
export LD_LIBRARYPATH=$LD_LIBRARY_PATH:$HOME/lib
Order is important, if you want your personal library searched BEFORE system libs reverse the order:
export LD_LIBRARYPATH=$HOME/lib:$LD_LIBRARY_PATH
Loading modules will automatically add necessary libraries to your LD_LIBRARY_PATH or LD_RUN_PATH.
lshw
This command will return some information for non-root users, although the information may be incomplete.
You may also run the following commands to get additional system information:
- lsblk: information and attached block devices (e.g. disks).
- lsscsi: information and devices on the scsi bus
- cat /proc/meminfo: information about system memory
- cat /proc/cpuinfo: information about the system cpus
- top/htop: broad information about processes and resource consumption
- dmesg: startup messages
firewalls
ICS managed computing employsiptables. Ports above 1024 on managed instructional and research Linux computing should be open to campus and VPN addresses. Please send requests for restricted ports, ports less than 1024, and other special requests to helpdesk@ics.uci.edu.
Note: Some local sudo users may have privilege to run /usr/sbin/iptables to open and close ports but any changes will be ephemeral. Please send a request to helpdesk@ics.uci.edu to make them permanent.
Reboot/Shutdown
Please reach out to helpdesk@ics.uci.edu if you believe a machine needs to be rebooted.
Shells
Sudo shell access is not allowed, it would allow users to circumvent the policies we have in place to protect security.
Sudoer Mess BoilerplatesTemplate
General
Our system reported that you ran the sudo command recently. I going to take this opportunity to share this wiki page that describes our sudoers policy, reasoning, and potential alternatives: https://wiki.ics.uci.edu/doku.php/policies:sudoers This page doesn't require you to login but links from this page may require ICS credentials to login.
Unnecessary Sudo Invocations
You are receiving this message because you recently tried running a command with sudo.
The command invoked does not require sudo privilege and can be run as your own user. Please let us know if there is a specific reason you felt that it was necessary to invoke via sudo. If we can figure out a non-sudo way to accomplish the same thing I will add it to our sudoers page:
https://wiki.ics.uci.edu/doku.php/policies:sudoers
Typically, the sudo command should be used exclusively to invoke commands not available to an ordinary user such as installing a package (apt or yum) or running a service on a privileged port (less than 1024).
Installation Alternatives
You are receiving this message because you recently tried running a command with sudo on an ICS managed server.
The following page details several alternative methods to accomplish what you were trying to do with the sudo command:
https://wiki.ics.uci.edu/doku.php/policies:sudoers
When using `pip install`, please use the –user switch to install into space that you own.
pip install --user nltk pip install --user requests
If you do not see a command in your path, please look at ICS modules with `module avail`:
https://wiki.ics.uci.edu/doku.php/commands:modules
Module ex.
$ module avail go ------------ /pkg/modules/modulefiles ------------ go/1.12.1 go/1.13.1 go/1.15.6 go/1.8.3 go/1.9 circinus-3 09:00:45 ~ $ module load go circinus-3 09:00:47 ~ $ which go /pkg/go/1.15.6/bin/go
You can determine what programs are available in your path using the which command:
circinus-3:~# which wget /usr/bin/wget circinus-3:~# which valgrind /usr/bin/valgrind circinus-3:~# which go /pkg/go/1.15.6/bin/go
Disk Usage
Hello <Name>,
We recorded the recent sudo activity on your account on <DATE>:
The command recorded was “sudo /bin/du -h”. Were you trying to get dis usage on the host? Were you just trying to look at your home directory but made a mistake? Please let us know what you were trying to do and maybe I could help.
Thank you and have a great day,
Shell and service invocations
Our system reported that you ran the sudo command recently. I'm going to take this opportunity to share this wiki page that describes our sudoers policy, reasoning, and potential alternatives:
https://wiki.ics.uci.edu/doku.php/policies:sudoers
This page doesn't require you to login but links from this page may require ICS credentials to login.
Some of the key points:
- Services on unprivileged ports (above 1024) should not run with sudo
- Sudo shell invocations are usuaully unecessary, we can find alternatives.
- Software not available via package managers may be installed in user space.
Please let us know if there is a specific challenge that caused you to use sudo and we can help figure out an alternative way to accomplish the same thing.
Troubleshooting
Q. sudo: error initializing audit plugin sudoers_audit
Symptoms: Sudo runs fine as root user but produces this error when run as a non-root user.
sudo: error initializing audit plugin sudoers_audit
Cause: User does not exist