A few notes on running Exchange in Microsoft Azure

Earlier this week Microsoft published a very interesting article: Exchange 2016 dev/test environment in Azure. I’m not a big fan of the how-to format of articles when the topic is around deployment because the guidance and specifics tend to change with later updates. There’s the risk of your article becoming obsolete sooner than planned.

The article contains an excellent real-world example of how to deploy a few virtual machines on Azure. I like how the author used PowerShell to configure the VM TCP/IP configuration and Install-ADDSForest to promote the server to domain controller.

Cool, let’s do this in production!

Although the article clearly states that the goal is the deploy Exchange for test and development, this article may spark new interest for people who would like to run Exchange in production too. The good news is that running Exchange in Azure is supported, but similar to running Exchange on other virtualization platforms some additional requirements must be met. The most significant requirement is actually mentioned in the article, in the section where storage is assigned for the Exchange VM:

This information can be found in the article Exchange 2016 virtualization as well:

Deployment of Exchange 2016 on Infrastructure-as-a-Service (IaaS) providers is supported if all supportability requirements are met. In the case of providers who are provisioning virtual machines, these requirements include ensuring that the hypervisor being used for Exchange virtual machines is fully supported, and that the infrastructure to be utilized by Exchange meets the performance requirements that were determined during the sizing process. Deployment on Microsoft Azure virtual machines is supported if all storage volumes used for Exchange databases and database transaction logs (including transport databases) are configured for Azure Premium Storage.

Running a single VM with the proper resources for Exchange 2016 24/7 in Azure is probably more expensive than you would think, with prices starting around € 500,- per month for the VM, two small premium storage disks and a Windows license.

While this falls outside of the scope of the article we’re discussing here, I would like to mention the requirement to use a smart host for outbound email delivery if you choose to run Exchange in Azure.

Make sure you do the math and understand the requirements before using this article as an alternative deployment plan for your next Exchange server. At this moment I am not aware of any organization running Exchange servers in Azure for production. If you are doing this, please reply in the comments.

But wait, there’s more

As I mentioned in my introduction Exchange installation how-to articles often contain wrong or obsolete information. Unfortunately that applies to this article too.

The latest version of Exchange, that’s actually great advice. Unfortunately the url https://go.microsoft.com/fwlink/p/?LinkId=747753 links to the download page for Exchange 2016 CU1. Not only is this version much older than the latest version, currently CU4, more importantly this version of Exchange is not even supported to run on Windows Server 2016. It was Exchange 2016 CU3 that introduced support for Windows Server 2016, although CU4 is recommended because a compatibility issue in Server 2016.

Exchange 2016 not supported for virtualization?

And while making some screenshots for this article I found another gem. On the page with the requirements for virtualization is this weird segment:

This is actually a left-over from the Exchange 2007 era, where this modernized virtualization policy was introduced. The requirement on that page was that Exchange 2007 SP1 was required and list of supported guest operating systems indicated that running a VM with Windows Server 2003 x64 was not supported.

With every new edition (2010, 2013, 2016) this page was copied without much changes. Instead of remove this no longer relevant section, the writers updated the section whit what they thought made sense. That’s what a VM running Exchange requires that it’s running Exchange 2010.

But I digress, back to Exchange 2016 now. The current version of this page does not include Windows Server 2016 as a supported operating system for running an Exchange 2016 VM. While this is obviously a mistake, technically speaking a virtualized Exchange 2016 server installed on Windows Server 2016 is currently not supported.

I found a doc error too, what should I do?

Shoot an email to Ex2013HelpFeedback@microsoft.com and make sure to include the url of the page, a quote and/or screenshot of the text you’re referring to and an explanation of why you think it’s in error. The team behind this alias is awesome and almost every time you will receive a response from an actual person.

MySQL Workbench and ClearDB: "SSL Connection error: unable to get private key file"

When trying to access your ClearDB hosted databases with MySQL Workbench you may run into this error message:

SSL Connection error: Unable to get private key

In the MySQL Workbench connection settings on the SSL tab we need to select three certificate files, the CA certificate, the client certificate and the client key file.

These files can be obtained from the ClearDB database dashboard:

The issue is being caused by the client private key file, the one with 'key' in the name. In order to allow Workbench to connect to the database the password needs to be stripped from this file first. One way to do this is to use OpenSSL:

openssl rsa -in c092bb5f82771e-key.pem -out c092bb5f82771e-key.pem-nopassw.pem

Now use the new file in the Workbench connection settings and try again, Workbench should now be able to connect to your database.

Source

How to remove the Change Password link from the Office 365 portal

Microsoft did an amazing job with their online services. I remember the web based management in the BPOS days, not a pleasant experience for both end users and admins. The current Office 365 portal is much improved and receives new updates every couple of months. Unfortunately there's one thing bothering many Office 365 admins and confusing even more end users.

In the Office 365 settings page (url: https://portal.office.com/EditProfile15.aspx), where the users can change their personal settings, is a Password link. This works great for cloud users, user accounts that have not been synced from an on-premises Active Directory, but not so well for synced users with either Password Synchronization or Identity Federation (ADFS).

When those type of users click this link to update their password the will receive an error:

Sorry, you can't change your password here. Follow the steps recommended by your organization or ask your admin for help.

So can we could remove the link, at least for our synced users? The answer is no, you can't currently do that. Recently I made a request through the Office 365 Support channel and so did other customers, leading to a Design Change Request (DRC). Unfortunately this DCR was declined by the Product Group because this change is not feasible with the current version of the portal.

If you want to be sure that the need for this change is under Microsoft's attention, please submit your feedback with the form on this page.

Azure AD Sync doesn't warn if scheduled task is enabled on Server 2008/2008 R2

After running the Azure Active Directory Sync Services (AADSync) configuration wizard, a scheduled task is created to run a sync job every three hours. When an admin starts the wizard again to make changed to the configuration a warning is thrown to disable the scheduled task and forced to restart the wizard. This is to prevent configuration changes to be made while an actual sync could be in progress. This check does not work with Server 2008 and Server 2008 R2.

Under the hood AADSync uses the Get-ScheduledTask cmdlet to determine the status of the scheduled task. Unfortunately this cmdlet was introduced in Server 2012 and Windows 8, it's not available on Server 2008 and Server 2008 R2. Both older versions of Windows Server are on the list of supported operating systems.

So what happens if you have AADSync installed on Server 2008 or 2008 R2 and start the wizard again? It does not warn you to disable the scheduled task first and allows you to change the configuration while a sync could be in progress. While the chances of that happening are relatively small with the three hour interval, this obviously is not something we want to happen. The application log shows event id 906 from source AzureActiveDirectoryDirectorySyncTool:

IsSchedulerEnabled() failed, assuming FALSE: Details: System.Management.Automation.CommandNotFoundException: The term 'Get-ScheduledTask' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.

The error message is self-explanatory, the IsScheduleEnabled function tries to use the Get-ScheduledTask cmdlet which is not available on this server. And the function assumes that the task is disabled, this is why we're no longer prevented from making configuration changes while the task is enabled. This behavior was noted with AADSync version 1.0.494.0501, the most recent version at this moment.

What does this mean for you if you're running AADSync on an older operating system? You should remember to verify that you disable the scheduled task before starting the configuration wizard, keep in mind that the wizard will not be able to check this and warn you if the task is still enabled.

Although Server 2008 and 2008 R2 are supported operating systems for AADSync I suspect Microsoft did not actually test the software on those operating systems. I brought the issue under their attention through Office 365 Support, an experience I wouldn't wish to my worst enemy. To be continued…

Concerns about using 3rd party add-ins in Azure

Recently I deployed a WordPress website from the Azure Marketplace. As with most Azure services the process of setting this up is very easy and a lot happens behind the screen.

One of the configuration items is a MySQL database provided by ClearDB.

So after you deploy the solution, the ClearDB database will be linked to your Azure Website. So far so good.

This morning I got a call from the web designer who said 'the server is down'. After some more questions I understood that the website was still available, but the connection with the database failed.

I checked the Azure Health dashboard and all checks were green. Which to be honest did not surprise me because my experience with the availability of Azure services has been very good until today.

Next stop was the Notifications section in the Azure Portal which had nothing too. In the mean time the web designer called and said the database was back but complained he had lost hours of work. At this point I was getting serious doubts about my decision to have my new company website built on Azure infrastructure.

At first I did not notice anything relevant on the ClearDB website until I read the FAQ again and noticed a link to their health information. It was immediately clear they had major issues in my region:

Unfortunately I discovered this hours after the incident happened and there was no mechanism in place for this information to reach me pro-actively.

I had a similar issue when I transferred some resources to a new Azure Subscription and someone deleted my SendGrid subscription. Had to visit the Azure Gallery, purchase the SendGrid service again and change the configuration of my Azure services to use the updated subscribtion.

Both incidents made me realize we still need to think of our services (website, email, ...) as a whole and include all dependent components in the picture. The fact that we are billed by Microsoft and have an manage the service from a single dashboard does not mean we have a single point of contact when it concerns support or health information.

Added to my (long) to-do list: Investigate if I can migrate from ClearDB MySQL to Azure SQL...