CHEC Library Systems virtualised

The Cape Higher Education Consortium (CHEC) manages Calico, a Library Management System for the four Western Cape Universities. This system ran on physical servers in the Internet Solutions Datacenter. The hardware became very expensive to maintain, and when three of the universities indicated they would be migrating to the cloud-based version of the software within the next 12-18 months, it became clear the current setup would need to be revisited.

CHEC approached UCT for assistance, and we proposed to virtualise the Calico infrastructure. Given the high availability requirements, the team (notably Waylon and Abraham), with some excellent project management from Leon Alexander, managed to do a P2V (Physical to Virtual) conversion, and ensured a successful cut-over this past weekend.

The development, production and warehouse environments are now running fully virtualised in a DMZ on ICTS’ VMware infrastructure.

Feedback from the libraries has been very positive, and we are happy to have been part of this project.

F5 High Availability

The below post by Waylon Horne:

Quick Steps to F5 BIG-IP LTM 11.x Virtual Edition High Availability

Acting as a reverse proxy and distributing network and application traffic amongst numerous servers is just the tip of the iceberg when it comes to the F5’s many features.
With today’s increasing demand for capacity, the F5 is becoming one of the most crucial devices in the networking jungle.

And with this demand, it’s only logical that it should be configured to be always available, hence the quick steps below to help you configure your F5 for HA.

 

 

1: Prerequisites:

  1. You must have a dedicated VLAN for HA.
  2. You must have Self-IP’s configured for both the HA VLAN network and Management network on both devices.
  3. These steps assume that you have setup the necessary defaults such as DNS, NTP, Admin Password, VLANS, etc.

2: HA VLAN setup:

  1. Login at https://YourLTMApplianceIP Enter the admin username/password.
  2. Go to Network > VLAN > VLAN List. Click
  3. Enter name
  4. Select 1.3 for interface, Tagging Click the Add button.
  5. Click
  6. Repeat the same steps as above on Appliance B.

3: Self IP setup:

  1. Go to Network > Self IPs. Click Create
  2. Type Name as SIP-HighAvailability. IP Address 0.0.1. Netmask as 255.255.255.0. VLAN as HighAvailability. Port lockdown Allow All. Select the Default Traffic Group (non-floating).
  3. Repeat the same steps as above on Appliance B using 0.0.2 as the IP.
  4. Click Finished.

4: Getting ready for HA setup:

  1. On Appliance A go to Device Management > Devices > Device Connectivity > Config Sync. Select the0.0.1(HighAvailability) VLAN IP.
  2. On Appliance A go to Device Management > Devices > Device Connectivity > Failover Network. Click Addunder Failover Unicast Configuration. Select your Management IP here.
  3. Repeat the same steps as above on Appliance B using Appliance B’s HA and Management IP.

5: HA cluster setup:

  1. On Appliance A go to Device Management > Device Trust > Peer List. Click Use the Management IP of Appliance B and admin username/password. Follow the rest of the steps
  2. Both appliances should now be paired with each other.
  3. On Appliance A go to Device Management > Device Groups. Click Create.
  4. Type name as SyncFailover to identify the device group which will participate in failover cluster.
  5. Group Type is Sync-Failover.
  6. Drag both IPs from right to left.
  7. Select Full Sync and Network Failover (leave Automatic Sync Unchecked).
  8. On Appliance A go to Device Management > Overview and select SyncFailover under Device Groups
  9. Select Appliance A under Devices, then select Sync Device to Group as well as the ticking the Overwrite Configuration check box. Click Sync.
  10. You HA cluster Setup is now complete. One appliance will now show as ACTIVE and the other one STANDBY.

More HA troubleshooting tips can be found on F5’s support KB article “SOL13946: Troubleshooting ConfigSync and device service clustering issues (11.x)

Cloud Apps: A brave new world…?

Since 2010 when we started moving our student email system to (then) Live@edu, I think we’ve learned quite a bit… I thought I’d share some of our experiences and (of course!) my own opinions, this being Part I….

The pace at which cloud providers (read: Microsoft & Google) are making changes and improvements to the functionality provided, is simply staggering compared to what an internal IT dept of a university could do. That does of course present us with a number of challenges, and I mention a couple of examples:

  • Integration with our on-premise systems: Cloud providers often integrate new products into the core suite (example: Yammer with Sharepoint Online), which opens up some real interesting conundrums. If we activate SSO (Single sign on) to a cloud application, some users might have subscribed to the product using their institutional email address, but with a different password. This happened with Google Docs when we on-boarded our @uct.ac.za domain into Google Apps for Education.
  • End-user support: It is very likely that a user will be using a specific function, maybe via a mobile app, way before any IT person is doing so. In the BYOD (bring your own device) world, users have access to Android & Apple apps, with associated access to back-end functionality not exposed by our on-premise systems. They expect us to support the application irrespective of origin, version or release date.
  • Training & documentation: The traditional model calls for us to have our training and documentation up-to-date and ready before a product or service is launched. This is becoming increasingly difficult, and we now end up linking to the vendor’s information directly. Of course this also means freeing up some time of technical writers 😉

So off we ride (fly?) into the cloud(s)…

 

Performance Tuning SQL Server – where to begin

When it comes to Performance Tuning, I have never come across DBA’s that have the same methods or step by step guide to performance tuning.  The question I get asked all the time , where to begin.

There is no right or wrong answer. If you are a  consultant and you get called in to a new environment, this is a Level 1 call, you don’t have the time to follow the full steps , you need a quick diagnostic short term solution,  Most DBA’s would start with steps 4 -5

For Real Time solutions , most DBA’s would look at what is running now , and then try to resolve.  One of the best ways of that is looking at your Waits in SQL Server. That would give you a clear indication of where the bottleneck is.

Most common top waits in SQL is related to Disk Subsystem. Before you start running perfmon and pulling out stats on your Disks presented to SQL, you need to ensure that your Data, Log, and Temp DB are  on separate Disks. This can impact SQL performance.

SQL will tell you in Activity Manager, which data files have high IO Latencies.  This can be related to index maintenance , missing indexes or poorly written queries.

Recently we had a performance issue on one on our core  Applications. It is highly used, average 384 batch requests per second. Wait stats showed High IO Latencies and Latching.
Perfmon showed the Latencies are fine for the different drives.  After further investigation, we found our normal maintenance plan which covered index maintenance was not sufficient for the increased high work load.

On another occasion ,  we did actually have a problem on the Disk Subsystems and the SAN vendors were called in to resolve.

If you no longer in crisis mode and want to get the best performance out of your SQL Server:

Operating System:
Must be patched with Service Packs. Ensure no errors.

SQL Infrastructure:
Data and Logs files are placed on separate disks with the correct raid levels.

Instance Configuration:
The defaults is generally fine. You may want to look at min & max memory settings. SQL will consume all the memory not leaving any for the operating system.
Maxdop is another option to look at. You can configure this to suit your environment, OLTP vs OLAP.
Default Index fill factor – default is 0. you can adjust to your environment.

Maintenance Plan:
Look at your index maintenance , update of stats , Integrity checks and backups

Blocking & Waits:
This will give you an indication of bottlenecks. A great tool to use.
See article by Paul Randall – Tell me where it hurts

Queries:
Look for long running queries , poor performing queries. Queries with missing indexes.

Databases:
Look at individual databases, tables, indexes, stored procedures and finally individual T-SQL queries.

Another Top SQL Server authority to follow is Pinal Dave – see his article on Performance Tuning

SAFIRE Project

The following post written by Donald Coetzee

South African Federated Identities for Research and Education

This project is currently underway where UCT is very involved in the move to federate with other institutions around South Africa. SAFIRE will provide a central authentication entry point to allow easier access to shared applications by authenticating users using their institution’s login credentials.
As an example: An application (Service Provider) called “filesender” is deployed at UCT, which provides for sharing of large files between users. Navigating to http://filesender.uct.ac.za, on the login screen, a redirection is performed to a “Discovery Service ”The user then clicks Logon, The page redirects to what is called “The Discovery Service”:
Fig1.1

The Discovery Service allows a user to specify which institution they are from , and this is where the “central” part in ‘central authentication entry point to allow easier access to shared applications by authenticating users using their institution’s login credentials’ comes in. Once the user chooses his/her ‘home institution’ the user is offered the opportunity to login to their home institution’s login service (Identity Provider).

fig1.2

Once the user is authenticated by the source institution, the application can be accessed and used.
fig1.3

In this case, Filesender as a federated application, can be used by an (authenticated) user from another (authorised) institution
MConf is another federated application hosted by SanRen, but UCT users are allowed to access it using their UCT credentials.
fig1.4

 

Mconf is an online meeting utility, that has also been joined to the federation, and when a user clicks ‘login via SAFIRE’:

fig1.5

If a user is already signed in to the Federation, there is no need to authenticate again.
As the federation matures, and more Applications (Service Providers), and Home institutions (Identity Providers) are added to the federation, the easier it will become for institutions in, and even those outside of South Africa to share resources, collaborate, and importantly:  share Research.
The project is currently in the deployment phase, and go-live will be communicated soon!

Visit from NWU staff

ICTS hosted five members from NWU IT. We shared ideas, solutions and (anecdotes), and I found it refreshing to see how similar challenges are being addressed in response to institutional requirements.

Some of the topics covered were around datacenters, compute, virtualisation, storage and Disaster recovery.

See what they are up to: http://www.nwu.ac.za/ithome

Email to Office 365…

We will be moving ICTS staff mailboxes from on-premise Exchange 2010/2013 servers to the Office 365 Datacenter (in Europe…), next week. We hope to use this as a final test before we start with the rest of the 9000-odd mailboxes.

A lot of work and preparation went into this project, and after this is done, the only box to be ticked will be to migrate away from Mailman as our mailing list software.

Meet the team…

 Name  Area of expertise Technologies
Eugene van Rooyen Line Manager  (Herding the sheep…)
 Donald Coetzee  Identity Management Novell IDM, SimpleSAMLPhp, E-Dir
 Durie Solomons  Identity Management Novell IDM, EDir
 Muzi Lubisi  Identity Management Novell IDM, Active Directory Federation Services
 Mohamed Ismail  Database Management Microsoft SQL DB & Reporting Services
 Ilyas Farrakhov  Database Management & Third Party Applications Microsoft SQL, MS CRM
 Marco Dippenaar  Microsoft Technologies  AD, ADFS, Sharepoint
 Pieter van Wyk  Microsoft Technologies Office 365, Exchange, Hyper-V
 Kwezi Plaatjie  Microsoft Technologies Exchange, WSUS
 Steve Ndabeni  Microsoft Technologies  Exchange, Office 365
Bongani Quwe  Datacenter & Facilities
Randy Thompson  Datacenter & Facilities
Waylon Horne  Virtualisation & Storage  VMware, Netapp
Brent Boswell  Virtualisation & Storage  VMware Linux
Mike Currin  Storage, Open Source Netapp, Linux
Stefan Coetzee  Virtualisation, Open Source, Security  Linux, Splunk, Openstack
Nelis Lamprecht  SAP ERP  Basis, Linux
Ian Josias  Peoplesoft Campus Solutions, DB2
Thembela Sonteya  SAP ERP, BI  Basis, Business Objects