======2021 Winter Maintenance Schedule ===== ==== ICS Server Update and Reboot Schedule ==== * Downtime Notice: Patch and Reboot ICS Servers * Date: March 22nd through March 25th, 2021 * Duration: Severs will be rebooted throughout the week. Servers providing critical services will be rebooted outside of normal work hours. The following ICS servers will be updated and reboot at the times noted below: ^ Hostname ^ Service ^ Maint Window Begin ^ Maint Window Duration | | adrian-monk-v11.ics.uci.edu | Puppet4 Configuration Server| 3/22/2021-0900 | 2 hours| | adrian-monk-v9.ics.uci.edu | Nagios Monitoring | 3/22/2021-900 | 2 hours| | broadchurch-v1.ics.uci.edu | Website: digphenotypes | 3/22/2021-0900 | 2 hours| | colin-baker-v0.ics.uci.edu | Puppet6 Configuration Server | 3/22/2021-0900 | 2 hours| | colin-baker-v1.ics.uci.edu | N/A | 3/22/2021-0900 | 2 hours| | colin-baker-v3.ics.uci.edu | N/A | 3/22/2021-0900 | 2 hours| | columbo-v2.ics.uci.edu | FTP Server (redundant) | 3/22/2021-0900 | 2 hours| | kate-beckett-v5.ics.uci.edu | Mail Delivery (Redundant)| 3/22/2021-0900 | 2 hours| | melvil-dewey-v1.ics.uci.edu | Helpdesk | 3/22/2021-0900 | 2 hours| | nancy-drew-v2.ics.uci.edu | OS Provisioning | 3/22/2021-0900 | 2 hours| | nancy-drew-v4.ics.uci.edu | Puppet Database | 3/22/2021-0900 | 2 hours| | richard-castle-v6.ics.uci.edu | Helpdesk | 3/22/2021-0900 | 2 hours| | rockford-v2.ics.uci.edu | LOM VPN | 3/22/2021-0900 | 2 hours| | rockford-v4.ics.uci.edu | Mail Service: IMAP| 3/22/2021-0900 | 2 hours| | rockford-v6.ics.uci.edu | Vault Secret Storage| 3/22/2021-0900 | 2 hours| | rockford-v8.ics.uci.edu | Authentication Server| 3/22/2021-0900 | 2 hours| | sakura-v4.ics.uci.edu | Logging: Logstash and Elk| 3/22/2021-0900 | 2 hours| | sasuke-v4.ics.uci.edu | Metrics: Graphana | 3/22/2021-0900 | 2 hours| | kate-beckett-v3.ics.uci.edu | Service host| 3/22/2021-1700 | 2 hours| | addison-v4.ics.uci.edu | Staff/Faculty Cycle Service | 3/22/2021-1700 | 2 hours| | adrian-monk-v1.ics.uci.edu | SGE Scheduler | 3/22/2021-1700 | 2 hours| | addison-v3.ics.uci.edu | Mail Service: IMAP| 3/22/2021-1700 | 2 hours| | nancy-drew-v5.ics.uci.edu | Mail Service: MailMan | 3/22/2021-1700 | 2 hours| | naruto-v4.ics.uci.edu | Metrics: Prometheus Collection | 3/22/2021-1700 | 2 hours| | paul-mcgann-v0.ics.uci.edu | SGE-Master | 3/22/2021-1700 | 2 hours| | richard-castle-v0.ics.uci.edu | Puppet4 Configuration Server | 3/22/2021-1700 | 2 hours| | richard-castle-v1.ics.uci.edu | Metrics: Ganglia| 3/22/2021-1700 | 2 hours| | richard-castle-v5.ics.uci.edu | Logging: Syslog Aggregator| 3/22/2021-1700 | 2 hours| | rockford-v1.ics.uci.edu | Staff/Faculty Cycler Server | 3/22/2021-1700 | 2 hours| ^ Hostname ^ Service ^ Maint Window Begin ^ Maint Window Duration | | addison-v9 | slurm-master | 3/22/2001-0900 | 2 hours | | castle-v4 | samba1 | 3/22/2001-0900 | 2 hours | | jo-grant | haproxy | 3/22/2001-0900 | 2 hours | | jodie-whittaker-v3 | | 3/22/2001-0900 | 2 hours | | peter-capaldi and its VMs | web services | 3/22/2001-0900 | 2 hours | | tom-baker-v0 | dhcp2 | 3/22/2001-0900 | 2 hours | | tom-baker-v1 | dcim | 3/22/2001-0900 | 2 hours | | tom-baker-v2 | | 3/22/2001-0900 | 2 hours | | liz-shaw | haproxy | 3/22/2001-1000 | 2 hours | | kate-beckett-v0 | samba2 | 3/22/2001-1000 | 2 hours | | kate-beckett-v4 | ns2 | 3/22/2001-1000 | 2 hours | | nancy-drew-v3 | phpadmin | 3/22/2001-1000 | 2 hours | | nancy-drew-v6 | const-esports | 3/22/2001-1000 | 2 hours | | sylvester-mccoy-v0 | dhcp1 | 3/22/2001-1000 | 2 hours | | sylvester-mccoy-v1 | | 3/22/2001-1000 | 2 hours | | nancy-drew-v0 | samba3 | 3/22/2001-1100 | 2 hours | | richard-castle-v2 | ns1 | 3/22/2001-1100 | 2 hours | | matt-smith and its VMs | web services | 3/22/2001-1300 | 2 hours | | addison-v5 | lbnamed | 3/25/2001-0900 | 2 hours | ^ Hostname ^ Service ^ Maint Window Begin ^ Maint Window Duration | | richard-castle-v3 | License Server | 3/22/2001-0900 | 2 hours | | adrian-monk-v2 | Cron Server | 3/22/2001-0900 | 2 hours | | adrian-monk-v10 | N/A | 3/22/2001-0900 | 2 hours | | kate-beckett-v6 | N/A | 3/22/2001-0900 | 2 hours | | melvil-dewey-v2 | Postgres Server | 3/22/2001-0900 | 2 hours | | rockford-v5 | N/A | 3/22/2001-0900 | 2 hours | | rockford-v9 | N/A | 3/22/2001-0900 | 2 hours | | rockford-v11 | N/A | 3/22/2001-0900 | 2 hours | | rockford-v13 | N/A | 3/22/2001-0900 | 2 hours | | rockford-v14 | N/A | 3/22/2001-0900 | 2 hours | | rockford-v15 | N/A | 3/22/2001-0900 | 2 hours | | rockford-v16 | N/A | 3/22/2001-0900 | 2 hours | | columbo-v0 | Mail Service: IMAP2 | 3/22/2001-0900 | 2 hours | | columbo-v1 | Logging: loghost1 | 3/22/2001-0900 | 2 hours | | columbo-v3 | N/A | 3/22/2001-0900 | 2 hours | | columbo-v4 | N/A | 3/22/2001-0900 | 2 hours | | columbo-v5 | N/A | 3/22/2001-0900 | 2 hours | | columbo-v6 | N/A | 3/22/2001-0900 | 2 hours | | columbo-v7 | N/A | 3/22/2001-0900 | 2 hours | | columbo-v11 | N/A | 3/22/2001-0900 | 2 hours | | datalab-storage1 | Data Storage Server | 3/23/2001-0900| 2 hours | | kraken | ZFS Server | 3/23/2001-0900| 2 hours | | hayes20t | ZFS Server | 3/29/2001-0900| 2 hours | ==== Openlab Reboot ==== * Downtime Notice: #78583: Patch and Reboot Openlab and Openlab8 Clusters * Date: March 25th , 2021 * Duration: 4 hours, beginning at 0800 **IMPACT**: All [[hardware:cluster:openlab|openlab]] and Openlab8 hosts will be patched, rebooted and moved from the puppet4 to puppet6 configuration servers. The hosts will be unavailable via SSH, SGE and Slurm during this period. SGE and Slurm spools will be turned off at end of business on 3/24. ==== Kubernetes Cluster Upgrade ==== * Downtime Notice: #78583: Upgrade Kubernetes Clusters * Date: March 22nd , 2021 * Duration: All week beginning at 0600 and lasting approximately 8 hours **IMPACT**: The Kubernetes clusters are built with redundant nodes so overall service should not be disrupted. Some users may notice pods or instances disconnecting and rebooting as they are migrating from one node to another. * [[https://phpmyadmin.ics.uci.edu]] * [[https://pgadmin.ics.uci.edu]] * [[https://pastebin.ics.uci.edu]] * [[https://speedtest.ics.uci.edu]] * [[https://grafana-infra-blue.ics.uci.edu]] * [[https://password.ics.uci.edu]] * [[https://prometheus-infra-blue.ics.uci.edu]] * [[https://hub.ics.uci.edu]] ==== Gitlab Upgrade ==== * Downtime Notice: #78583: Upgrade Gitlab Server * Date: March 24nd , 2021 * Duration: 0800 to 12:00 **IMPACT**: The site gitlab.ics.uci.edu will not be available during upgrade window. Repositories will not be available to push/pull from. This will not impact gitlab setup for individual classes. ==== Engineering Tower Generator ATS repair On Jan 30th ==== * Downtime Notice: #78155 : ICS Data Center Power Event: Engineering Tower Generator ATS repair * Date: January 30th, 2020 * Duration: Beginning at 0600 and lasting approximately 8 hours Campus facilities scheduling an electrical power shut down for the ICS building on 1/30/2021 in order to effect repairs on a failed generator automatic transfer switch.    **Network**: Power will be temporarily reconfigured to make sure all ICS network equipment remains powered during this period. **Computing**: We do not anticipate downtime on any ICS computing during this period. **Environment**: One of three 20 ton air conditioners will be unavailable during this period. Portable cooling units will be used to supplement the remaining air conditioners. If the portable cooling units prove unable to maintain a reasonable operating temperature we will begin shutting down non-critical infrastructure followed by hosts generated the greatest amount of heat (e.g. GPU clusters). **OIT**: OIT will be mirroring our efforts to make sure that their equipment in the building is available during this period. Although we will do our best to avoid downtime during this period, there are many factors that can intersect unexpectedly to cause an outage. Please send email to helpdesk@ics.uci.edu if you are concerned that this work will cause a hardship on your research and we will work with campus facilities to do our best to accommodate. ==== Last Quarter ==== * [[:announce:fall-2020|Fall Quarter 2020 Announcements]] * [[:projects:maint-fall-2020|Fall Quarter 2020 Maintenance Schedule]]