Tag Archives: storage

Central Storage Maintenance and Potential Unavailability

WHAT is going on:

Please be advised that IT Infrastructure Services will be conducting central storage maintenance to continue to improve the stability and performance of the storage systems.  This maintenance may cause brief intermittent outages of access to central storage, so please attempt to minimize utilizing your home directory or network file shares during the maintenance window described below.

WHEN it will happen:

The scheduled storage maintenance will be from Friday May 9, 2014 at 8:00pm to Saturday May 10, 2014 at 8:00am.

WHY are we doing this:

The performance and stability of the new central storage system is our top concern.  We sincerely feel that you – the faculty, staff, and students – deserve the most affordable, stable, versatile, and highly-performing storage system available.  IT management and staff have been working closely with our storage vendor and together we have developed a detailed quality improvement plan.  This maintenance window will allow us to implement the many improvements in this plan with the least amount of negative impact on users.

WHOM will this affect:

The storage system undergoing maintenance affects all campus home directories, most academic and research shares, and most administrative data.  All students, staff, and faculty may see intermittent access loss to their home directories during this maintenance window. This maintenance will not affect storage attached to the Superior High Performance Cluster nor any other locally-attached storage.

If you have any questions, comments or concerns, please contact IT User Services at (906) 487-1111 or email it-help@mtu.edu.

Status of Storage Issue

Thank you for your continued patience and understanding while we make progress bringing you the quality centralized storage services you deserve. Our centralized storage plays a critical role in many processes – from logging in on your computer to printing – and is necessary for the healthy operation of applications from BANNER to your browser. The scale of the service disruption  is not lost on me or my staff and again, I offer my apology for the great inconvenience this reduction in service has caused.

Over the last week my staff has tirelessly worked to stabilize the performance of the central storage to the point that it is acceptably useable. They have tweaked and tuned everything they could and have succeeded in stabilizing the patient. To continue the medical analogy, we have moved the patient from the ICU to the general care ward and we continue to actively monitor the health of the system.

We are not through. I am not satisfied with “acceptable.” I only will be satisfied when the new storage equipment operates in a superior fashion, as originally specified, and as rightfully expected. Working aggressively with our storage vendor we have arranged for additional storage equipment to be delivered and installed as soon as is possible. We expect the additional equipment to be here within the week, and you should see a lasting improvement in your centralized storage services as soon as we bring the first piece of equipment online.

My goal, from this recent storage upgrade, was to provide you the most capable, reliable, flexible, and highly performing storage for your campus computing needs. The additional equipment and the continued work of my staff and all those involved in bringing this additional hardware to bear on this problem will meet this goal.

In closing I would also like to thank the campus community for the many supportive words and emails during this difficult time. So rare a thing, this fellowship during adversity, is the hallmark of Michigan Tech students, faculty, and staff.

Status of Ongoing Storage Issues

First, let me offer my deepest apology for the disruption in central storage services that is impacting the ability to use campus computing resources.

After the home directory upgrade was complete, it wasn’t until enough load was put on the new storage system that an unforeseen performance issue became apparent. The new system was specified and designed to be at an order of magnitude more powerful, flexible, and high-performing, and thus far, it has failed to do so. Since the discovery of the performance issue on Monday the IT Storage Services team and I have been actively working to resolve the problem, and we have engaged top level engineers from the storage vendor. In turn, they have escalated our support case as high as possible within their organization. We are aggressively pushing our vendor for a rapid solution and will not ease up on our efforts until the performance problems are eliminated.

In the event that a rapid solution is not forthcoming, we have a roll-back plan in reserve. This would mean an additional complete home directory outage, that would span roughly 24 to 36 hours. In the event that the decision to roll back is made, Information Technology will send out an additional announcement.

Please know that we are working around the clock to resolve this issue and sincerely appreciate your patience during this difficult period. If you have any questions, comments, or concerns please do not hesitate to contact me directly at ccwill@mtu.edu.

Cayce Will
Director of IT Infrastructure Services