Thursday, December 31, 2015

Do not convert synced mailboxes to shared in a hybrid environment

Recently I ran into an issue with shared mailboxes in Exchange Online. In the Office 365 Admin portal I created a custom view in the Active users section to track issues with our licensing process:

Some users were displayed here which where moved from on-premises as regular mailboxes and then converted to shared with the button in EAC. Which is odd because a license isn’t required for a shared mailbox.

At a first glance everything seemed okay, the mailbox moved to the Shared view in EAC. The only thing off was the fact that LicenseReconciliationNeeded was set to True for this user which is why it showed up as mailbox without a license in the portal.

Next I compared the mailbox properties as well as the on-premises AD attributes with another shared mailbox, that happened to be shared before it was moved to Exchange Online.

So for the shared mailbox that was converted before the move msExchRemoteRecipientType was set to 100 (Shared mailbox in Exchange Online) while the other still displayed a value of 4 (Migrated mailbox).


Next step was to involve Microsoft Premium Support. Their first recommendation was to change the value from 4 to 100 in the on-premises AD and have it synced back to Azure AD. This didn’t make sense to me and by that time I already discovered that I could easily reproduce the issue in the test environment by converting a moved mailbox with the button in EAC so it was not caused by an issue in our on-premises AD.

Next the engineer provided the following KB article and explained that the behavior I’m seeing is ‘by design’: Shared mailboxes are unexpectedly converted to user mailboxes after directory synchronization runs in an Exchange hybrid deployment.

The article describes a scenario where a mailbox has been moved to Exchange Online, is converted to Shared and with the next directory sync is converted back to regular. Interestingly that last part is not happening in our environments and I was not able to reproduce the described behavior, not with a highly customized FIM nor with a default AAD Connect sync engine.

Anyway, accoring to the article it is not supported to convert a regular mailbox for a synced user to the shared mailbox type. The technical explanation is that directory synchronization currently doesn’t support the syncing back of all attributes that change during the conversion.

Microsoft recommends to move the mailbox to on-premises, convert it to shared and then move it back to Exchange Online. For some customers this may present an acceptable work-around, for others it is cumbersome and requires additional planning and communication to end-users.

And the real question of course is why Microsoft allows their customers to convert a synced mailbox to shared when the underlying technical issue can result in the mailbox being deleted after the grace period ends. I left feedback through the support channel and on the Office 365 Uservoice site. Please vote here if you agree this should change: Support conversion to shared mailbox for synced users

Thursday, December 17, 2015

My take on the Star Wars The force awakens story line

Just kidding, this is a test post with version 5.1.2 of Open Live Writer. In case you missed it, Open Live Writer is the open source successor of the popular blogging tool Windows Live Writer. WLW once was part of the Windows Live Essentials bundle of tools and has a large share of fans, even though development and improvements are absent since 2012.

Read more about OLW here:

Tuesday, December 15, 2015

Exchange 2013 CU11 and Exchange 2007/2010 Update Rollups released

Microsoft today announced the release of the 11th Cumulative Update for Exchange 2013. That's not all, Exchange 2010 SP3 receives Update Rollup 12 and there's even one for Exchange 2007: UR 18. The latter is basically a DST update so don't expect any other issues to be fixed after you install this update.

Click the link to read more. One more thing, make sure you read this article on the Exchange Team Blog too: Exchange Management Shell and Mailbox Anchoring. This change in behavior may impact the way you manage Exchange with PowerShell. There's been some concern in the community about the benefits of this change, some people suggest that it introduces more problems than is solves. More about this subject on Reddit: Major change to how PowerShell works with Exchange

Now the wait is for Exchange 2016 CU1 which will be released with the next set of semi quarterly updates at the end of 2016Q1.

Monday, December 14, 2015

December 8, 2015, update for Outlook 2010 causes Outlook to start in safe mode

I noticed several people reporting weird issues with their Outlook 2010 clients. For instance the loss of signature and other settings. The cause seems to be installation of update KB3114409. This update adds a new feature:

Adds administrative support to prevent Outlook 2010 from booting into safe mode. Administrators set this function in some scenarios when they have add-ins that must be enabled.
Admins can use a registry key or GPO to prevent the users from starting Outlook in safe mode. The issue is, without those controls Outlook now starts in safe mode by default.

The fix is relatively easy, simply uninstall update KB3114409. Alternatively admins can use the aforementioned registry key to control Outlooks behavior. Some smart people on Reddit figured out the correct entries for both x86 and x64 systems: KB3114409 breaking Office 2010, forcing it to start in Safe Mode, which is the exact opposite of what's supposed to happen.

Microsoft pulled the update so I expect no new occurrences. Anyway, this is why we roll-out patches to a small test group before rolling them out to the entire enterprise, don't we?

Wednesday, December 2, 2015

What every Office 365 admin should know about supported Outlook versions

Working with customers that are beginning to consume Office 365 services I see a lot of confusion about whether their old Office version should work with Office 365. Especially when we’re talking Office 2010 or even 2007. What certainly not helps is that Microsoft restructured their Office 365 documentation in such a manner that it is hard to find all relevant information.

The general Office 365 for Business requirements states the following:

Make sure that your Office clients are compatible with Office 365. Office 365 works with any version of Office in mainstream support: the latest version of Office, Office 2013, and Office 2011 for Mac. Previous versions of Office clients, such as Office 2010 and Office 2007, may work with Office 365 with reduced functionality.

The requirements for Exchange Online are more restrictive and definitely clearer, although they are hidden in the Exchange 2016 system requirements page:

Exchange 2016 and Exchange Online support the following versions of Outlook:

  • Outlook 2016
  • Outlook 2013
  • Outlook 2010 with KB2965295
  • Outlook for Mac for Office 365
  • Outlook for Mac 2011

Outlook clients earlier than Outlook 2010 are not supported. Email clients on Mac operating systems that require DAV, such as Entourage 2008 for Mac RTM and Entourage 2004, are not supported.

So your users are still on an older Outlook 2010 build without SP2 and the April 15, 2014 update? Not supported. Outlook 2007? Not supported. From experience I can tell that these requirements are not a joke, your users will not be able to connect to Exchange Online with Outlook 2010 RTM. Another customer saw Outlook 2007 crashing after moving the mailboxes to Exchange Online and was able to work-around this by removing the language pack.

So if we like it or not, consuming a cloud service equals losing control about what client software you’re using. It’s the cloud provider that dictates the supported versions and when they think you should upgrade. That brings me to the final point, that these requirements not static and change over time. So make sure you visit the sources regularly to see of there have been updates.

Monday, November 2, 2015

Install an older Exchange version after last server was removed

Consider a scenario where an Exchange environment was upgraded from let’s say Exchange 2007 to Exchange 2010 (I know, old versions). Some time after the admin removed the last Exchange 2007 server they find out that they a LOB application is not working correctly and they need to add a server with Exchange 2007 to allow it working again. The question is: can we add an older Exchange version after we removed the last server from the environment?

A community member on Reddit’s ExchangeServer community recently stated that this would not be possible:

…you won't be able to install a new 2007 server into a forest with 2010 servers once you've removed the last 2007 server…

In a follow-up discussion the poster referred to this article as his source:

Question: Is it possible to install Exchange 2003 or 2007 in a pure Exchange 2010 organization?
Answer: [...] If you have transitioned from Exchange 2007 to Exchange 2010 and the last Exchange 2007 server has already been decommissioned, the answer is again no. You will not be able to install Exchange 2007 at a later time in this organization because it’s now considered a pure Exchange 2010 organization.


This excerpt is from an article in the TechNet Magazine and was written by a well-known Exchange export so I don’t blame the Reddit poster at all. However, with my knowledge of how Exchange works I could not think of any reason why this shouldn’t work and this is the first time I heard of it.

My position on this subject is that there is no reason why you should not be able to add an Exchange 2007 server in a Server 2010 forest after the last Exchange 2007 was removed.

From a technical perspective there’s nothing special happening after you remove the last Exchange 2007 server. The Exchange environment does not keep count of the number of Exchange 2007 servers and there’s no process that implements a blocker when the count reaches <1. Also there’s no such thing as a ‘pure Exchange 2010 organization’, unless you want to say that there are no other Exchange versions than 2010 in the environment.

Time to validate this in a lab:

  1. Create a domain controller for a new forest, DFL and FFL set to Windows Server 2003
  2. Add a server with Exchange 2007 SP3
  3. Add a server with Exchange 2010 SP3
  4. Move the mailboxes and OAB generation to Exchange 2010
  5. Remove the Exchange 2007 mailbox database and storage group
  6. Uninstall Exchange 2007
  7. Add a server with Exchange 2007

As expected I was able to add the Exchange 2007 server to the environment without any issues.

Now in a real world environment there can be other factors involved but’s it’s good to understand that your Exchange organization does not add a block when you remove the last server of a certain Exchange version.

Friday, October 30, 2015

Is it possible to install Exchange 2016 on a domain controller?

Can we?

This is a question that comes up every now and then in the technical communities I’m involved in. Is it possible and supported to install Exchange on a domain controller? Often answered incorrectly with a variant of “not supported, unless you run SBS”. The official answer is a bit different: it is supported to install Exchange on a domain controller.

In fact, this is supported since Exchange 2000 and has not changed for every new version up to the recently released Exchange 2016. As we can find in the Exchange 2016 System Requirements page:


Installing Exchange 2016 on directory servers
For security and performance reasons, we recommend that you install Exchange 2016 only on member servers and not on Active Directory directory servers. However, you can't run DCPromo on a computer running Exchange 2016. After Exchange 2016 is installed, changing its role from a member server to a directory server, or vice versa, isn't supported.

So should we?

Supported but not recommended, why is that? There are multiple reasons why you should consider to install Exchange on a member servers.

Firstly both Exchange and an Active Directory domain controller are resource-intensive tasks. For instance, both need to cache data in memory for best performance and will try to claim resources to do so. Running both tasks on the same computer may impact the performance and end-user experience for both AD and Exchange.

Exchange supports an RBAC model which allows an organization to deploy the least amount of privileges to their admins to do their work. However, when an admin needs local administrative access to server which has Exchange installed he gains administrative access to Active Directory too. This would enable the admin to elevate their own privileges which of course is a security risk.

A domain controller with Exchange installed cannot be demoted to remove AD from the computer as long as Exchange is installed, the last version that did support this was Exchange 2000. This may mean that at some later time an organization may find they can’t remove domain controllers from their environment without removing Exchange from the server first. In the recent past this is something that organizations often encountered when they needed to remove Windows Server 2003 domain controllers in order to be able to raise the domain or forest functional level. By the way, you also can’t promote a server with Exchange already installed but that is a much less common scenario.

I briefly touched the subject of security when discussing your admins working on this server, security needs considered with user access in mind too. Some firewall ports need to be forwarded to this server to accept email and give access to OWA/Outlook on the web, ActiveSync, Outlook Anywhere and maybe even legacy protocols as POP3 and IMAP4. Nowadays Exchange is considered secure enough to expose it to the internet directly but do you want to allow hackers to target your domain controller too? Opening firewall ports to allow access from the internet to your domain controller is not something I would call a security best practice.

There’s even more reasons why you should not install on a Exchange server but I won’t do into too much details here. Use your favorite search engine to do a search on this topic and you will find a ton of discussions about this subject.

Yeah, but what if…?

I am aware that for some very small organizations it may not make sense to purchase additional hardware and Windows Server licenses. Certainly not from a cost perspective. With a small load it’s unlikely that performance is going to be an issue and many small business owners will consider the security implications something for large enterprises to be concerned about. After all, Microsoft until recently sold the Small Business Server products that did install Active Directory, Exchange, SharePoint, ISA/TMG and even SQL on a single box. If that was an acceptable solution, why shouldn’t we do that with just AD and Exchange?

Running Exchange on a domain controller is a supported solution and as long as you understand the impact, go ahead and deploy that single server. When performance, security and flexibility play a role and the budget or infrastructure allows for, it’s recommended to deploy a new Windows server and install Exchange on that server.

Wednesday, October 28, 2015

How to remove the Exchange Autodiscover SCP

Consider a scenario where you moved all your user’s mailboxes to Exchange Online, for instance after a cutover or staged migration and want to remove any dependencies on the local Exchange server. You may find that Outlook still connects to the local Exchange server for Autodiscover lookups, this is because Outlook is hard-coded to query an AD Service Connection Point to locate a server with the Autodiscover service. When this fails Outlook falls back to the next DNS based methods or uses a local XML file.

Exchange Management Shell

There are multiple ways to prevent Outlook from contacting the local Exchange server first, some of them make more sense than others. The preferred way is to use the Exchange Management Shell to clear the entry for the Client Access server from the SCP.

Set-ClientAccessServer –Identity ServerName -AutoDiscoverServiceInternalUri $null


This removes the SCP entry for this Exchange server.


If the above method can no longer be used a low level AD editor as EDSIEdit can be used to remove the SCP manually. The full path of the SCP is:

CN=ServerName,CN=Autodiscover,CN=Protocols,CN=ServerName,CN=Servers,CN=Exchange Administrative Group (FYDIBOHF23SPDLT),CN=Administrative Groups,CN=OrganizationName,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=DomainName,DC=Suffix

This object to remove has the Class type serviceConnectionPoint.


Alternative methods

One of the above steps is basically all you need to do. Alternatives include adding an ExcludeScpLookup value to the \Autodiscover registry key (KB article) and some articles even let you remove the Autodiscover virtual directory from IIS. This will of course make Outlook unable to query the local Exchange server for Autodiscover but why should you if you can simply remove the SCP.

How to verify?

As always, the proof is in the pudding. Use the Outlook test E-mail AutoConfiguration feature to verify the clients behavior. We’re specifically interested in the Log tab where we should see that Outlook is no longer able to query the SCP to obtain the Autodiscover url.

To start the Test E-mail AutoConfiguration tool, follow these steps:

  1. Start Outlook.
  2. Hold down the Ctrl key, right-click the Outlook icon in the notification area, and then click Test E-mail AutoConfiguration.
  3. Verify that the correct email address is in the E-mail Address box.
  4. In the Test E-mail AutoConfiguration window, click to clear the Use Guessmart check box and the Secure Guessmart Authentication check box.
  5. Click to select the Use AutoDiscover check box, and then click Test.

Earlier I wrote a short article about this tool, unfortunately available in Dutch only: Autodiscover testen met Outlook. But you’ll get the general idea. Focus on the first or third tab when you’re interested in the Autodiscover results, read the Log tab when you're interested in the Autodiscover process.

Wednesday, October 21, 2015

MySQL Workbench and ClearDB: "SSL Connection error: unable to get private key file"

When trying to access your ClearDB hosted databases with MySQL Workbench you may run into this error message:


SSL Connection error: Unable to get private key

In the MySQL Workbench connection settings on the SSL tab we need to select three certificate files, the CA certificate, the client certificate and the client key file.


These files can be obtained from the ClearDB database dashboard:


The issue is being caused by the client private key file, the one with 'key' in the name. In order to allow Workbench to connect to the database the password needs to be stripped from this file first. One way to do this is to use OpenSSL:

openssl rsa -in c092bb5f82771e-key.pem -out c092bb5f82771e-key.pem-nopassw.pem

Now use the new file in the Workbench connection settings and try again, Workbench should now be able to connect to your database.


Tuesday, October 13, 2015

How to view the last monitor response in NetScaler 10.5

When a monitor fails it can be very helpful to check the last result it received. The NetScaler 10.5 web interface does present this information, but you have to know where to look:

Open de properties of the Service Group, expand the Service Group Members, select a Service and click Monitor Details.

The result can be found under the Last Response column: