Project Honolulu and Exchange servers

Recently Microsoft announced a new web-based server management console, it’s called Project Honolulu. Looking at the documentation and screenshots this is going to be a huge step forward from the native sever management consoles that we have seen from Server 2008-2016.

A technical preview of Project Honolulu is already available for download: https://aka.ms/HonoluluDownload Project Honolulu can be used to manage servers with Windows Server 2012, 2012 R2 and Server 2016. However, there is a requirement to install PowerShell v5 on those older servers. From the documentation:

Honolulu requires PowerShell features that are not included in Windows Server 2012 and 2012 R2. If you will manage Windows Server 2012 or 2012 R2 with Honolulu, you will need to install Windows Management Framework (WMF) version 5.0 or higher on those servers.

Unfortunately Microsoft does not support installing newer versions of WMF on servers that have Exchange installed. The Exchange Supportability Matrix says the following about the supported versions of WMF on servers running Exchange:

Version of Windows Management Framework built into the release of Windows Server you're installing Exchange on.

This means that installing PowerShell 5 on Server 2012 or Server 2012 R2 is not supported for servers running Exchange, effectively making it impossible to use Project Honolulu to manage Exchange servers. The only exception is Exchange 2016 running on Server 2016 because this OS was released with PowerShell v5 natively and meets the Project Honolulu requirements.

Exchange Hybrid? Microsoft has no plans to make creating shared mailboxes easy.

In two earlier posts (one, two) I wrote about the limited options to provision shared mailboxes in a hybrid environment. Or more specific, in an environment with directory synchronization. In short, it’s not possible to create shared mailboxes or convert regular mailboxes to shared in Exchange Online.

While both New-RemoteMailbox and Set-RemoteMailbox support the -Type parameter,  but it will only accept Regular, Room or Equipment as values and not Shared. We asked Microsoft to reconsider and add support to create remote shared mailboxes. Unfortunately the Design Change Request (DCR) was rejected. No specific reason was given but indicated was that our request was the first and only ask for this feature.

Earlier, when we suggested to remove the Convert to shared button from EAC Microsoft stated they considered customers wanting to convert a mailbox to shared a ‘niche scenario’. If you disagree and think customers should be able to provision and convert to shared mailboxes, make sure to let Microsoft know. Managed customers should ask their TAM, for smaller customers I’m afraid they need to burn a $499 support call as I’m not aware of another channel to add your request to Microsoft’s database.

For now this means that new shared mailboxes need to be provisioned on-premises and then be moved to Exchange Online. To convert a mailbox in Exchange Online we need to move it back to on-premises, convert the mailbox and then move it to Exchange Online again. Or read my work-around: Convert a user mailbox to shared in a hybrid environment.

PowerShell 5.0 re-released. Do not install on Exchange!

Two weeks ago Microsoft decided to offer the latest version of .Net Framework as a recommended update. Many Exchange admins found out the hard way that it’s not wise to install every single update without checking if it is actually supported to run them in combination with Exchange. In case of .Net Framework 4.6.1 there were in fact known issues, as some people soon discovered.

Today Microsoft re-released Windows Management Framework (WMF) 5.0 RTM. WMF included PowerShell 5.0 which brings many new features. Advanced administrators are probably looking forward to install WMF 5.0 on all their systems as soon as possible. But don’t do that, not before you’re absolutely sure that it is supported with Exchange.

This information can be found in the Exchange Server Supportability Matrix, one of the most important Exchange resources that’s often overlooked. In the ESSM we find for instance that .Net Framework 4.6 is not supported:

And the same applies to WMF 5.0:

And for customers with Outlook 2007 who consider Exchange 2016:

By the way, it’s perfectly fine to use PowerShell (WMF) 5.0 to connect to Exchange Online. In fact, if you’re on the November update of Windows 10 (Version 1511) this means that PowerShell 5.0 is already installed on your system.

Convert a user mailbox to shared in a hybrid environment.

In a hybrid environment it’s not supported to convert user mailboxes to regular, even there is a link in EAC to do this. It seems to work, but the changes that are made in Exchange Online won’t properly sync back to on-premises. I wrote about this in an earlier post: Do not convert synced mailboxes to shared in a hybrid environment.

After that I kept working with Microsoft to obtain a better understanding of the issue and ultimately develop a process to do this conversion.

Disclaimer: This process was developed in a lab environment under the guidance of Microsoft Premier Support. Before doing this in your environment, make sure you check with your Microsoft contact if they support this procedure until any official guidance has been published.

To convert a mailbox to shared, we need to perform three steps:

• Convert the mailbox to shared in Exchange Online
• Modify the on-premises AD attributes accordingly
• Revoke the Exchange Online license

In Exchange Online simply convert the mailbox to the correct type:

Set-Mailbox MyMailbox -Type Shared

Now in Active Directory Users and Computers, make sure you enabled Advanced Features under the View menu option. Next navigate to the AD object (mail user), open it’s properties and go to the Attribute Editor tab.

Tip: Write down the values before making any changes. Or even better, dump all AD attributes and their values to a text file:
Get-ADUser MyMailbox -Properties * > before.txt.

Now update the following attributes with these values:

• msExchRemoteRecipientType: 100
• msExchRecipientTypeDetails: 34359738368

Last step is to revoke the Exchange Online license. This is optional but in most cases something you want to do as a shared mailbox does not require a license. Simply use the Office 365 portal and find the user under Active Users. Remove the Exchange Online license.

After we revoked the license it’s important to validate the license status in Azure AD:

Get-MSOLUser -UserPrincipalName MyMailbox@mydomain.com | fl *lic*

Pay attention to the LicenseReconciliationNeeded attribute, this should be False. If LicenseReconciliationNeeded returns True Exchange Online thinks this mailbox requires a license and entered the 30 day grace period. A fix

Install an older Exchange version after last server was removed

Consider a scenario where an Exchange environment was upgraded from let’s say Exchange 2007 to Exchange 2010 (I know, old versions). Some time after the admin removed the last Exchange 2007 server they find out that they a LOB application is not working correctly and they need to add a server with Exchange 2007 to allow it working again. The question is: can we add an older Exchange version after we removed the last server from the environment?

A community member on Reddit’s ExchangeServer community recently stated that this would not be possible:

…you won't be able to install a new 2007 server into a forest with 2010 servers once you've removed the last 2007 server…

In a follow-up discussion the poster referred to this article as his source:

Question: Is it possible to install Exchange 2003 or 2007 in a pure Exchange 2010 organization?
Answer: [...] If you have transitioned from Exchange 2007 to Exchange 2010 and the last Exchange 2007 server has already been decommissioned, the answer is again no. You will not be able to install Exchange 2007 at a later time in this organization because it’s now considered a pure Exchange 2010 organization.

This excerpt is from an article in the TechNet Magazine and was written by a well-known Exchange export so I don’t blame the Reddit poster at all. However, with my knowledge of how Exchange works I could not think of any reason why this shouldn’t work and this is the first time I heard of it.

My position on this subject is that there is no reason why you should not be able to add an Exchange 2007 server in a Server 2010 forest after the last Exchange 2007 was removed.

From a technical perspective there’s nothing special happening after you remove the last Exchange 2007 server. The Exchange environment does not keep count of the number of Exchange 2007 servers and there’s no process that implements a blocker when the count reaches <1. Also there’s no such thing as a ‘pure Exchange 2010 organization’, unless you want to say that there are no other Exchange versions than 2010 in the environment.

Time to validate this in a lab:

1. Create a domain controller for a new forest, DFL and FFL set to Windows Server 2003
2. Add a server with Exchange 2007 SP3
3. Add a server with Exchange 2010 SP3
4. Move the mailboxes and OAB generation to Exchange 2010
5. Remove the Exchange 2007 mailbox database and storage group
6. Uninstall Exchange 2007
7. Add a server with Exchange 2007

As expected I was able to add the Exchange 2007 server to the environment without any issues.

Now in a real world environment there can be other factors involved but’s it’s good to understand that your Exchange organization does not add a block when you remove the last server of a certain Exchange version.

Is it possible to install Exchange 2016 on a domain controller?

Can we?

This is a question that comes up every now and then in the technical communities I’m involved in. Is it possible and supported to install Exchange on a domain controller? Often answered incorrectly with a variant of “not supported, unless you run SBS”. The official answer is a bit different: it is supported to install Exchange on a domain controller.

In fact, this is supported since Exchange 2000 and has not changed for every new version up to the recently released Exchange 2016. As we can find in the Exchange 2016 System Requirements page:

Installing Exchange 2016 on directory servers
For security and performance reasons, we recommend that you install Exchange 2016 only on member servers and not on Active Directory directory servers. However, you can't run DCPromo on a computer running Exchange 2016. After Exchange 2016 is installed, changing its role from a member server to a directory server, or vice versa, isn't supported.

So should we?

Supported but not recommended, why is that? There are multiple reasons why you should consider to install Exchange on a member servers.

Firstly both Exchange and an Active Directory domain controller are resource-intensive tasks. For instance, both need to cache data in memory for best performance and will try to claim resources to do so. Running both tasks on the same computer may impact the performance and end-user experience for both AD and Exchange.

Exchange supports an RBAC model which allows an organization to deploy the least amount of privileges to their admins to do their work. However, when an admin needs local administrative access to server which has Exchange installed he gains administrative access to Active Directory too. This would enable the admin to elevate their own privileges which of course is a security risk.

A domain controller with Exchange installed cannot be demoted to remove AD from the computer as long as Exchange is installed, the last version that did support this was Exchange 2000. This may mean that at some later time an organization may find they can’t remove domain controllers from their environment without removing Exchange from the server first. In the recent past this is something that organizations often encountered when they needed to remove Windows Server 2003 domain controllers in order to be able to raise the domain or forest functional level. By the way, you also can’t promote a server with Exchange already installed but that is a much less common scenario.

I briefly touched the subject of security when discussing your admins working on this server, security needs considered with user access in mind too. Some firewall ports need to be forwarded to this server to accept email and give access to OWA/Outlook on the web, ActiveSync, Outlook Anywhere and maybe even legacy protocols as POP3 and IMAP4. Nowadays Exchange is considered secure enough to expose it to the internet directly but do you want to allow hackers to target your domain controller too? Opening firewall ports to allow access from the internet to your domain controller is not something I would call a security best practice.

There’s even more reasons why you should not install on a Exchange server but I won’t do into too much details here. Use your favorite search engine to do a search on this topic and you will find a ton of discussions about this subject.

Yeah, but what if…?

I am aware that for some very small organizations it may not make sense to purchase additional hardware and Windows Server licenses. Certainly not from a cost perspective. With a small load it’s unlikely that performance is going to be an issue and many small business owners will consider the security implications something for large enterprises to be concerned about. After all, Microsoft until recently sold the Small Business Server products that did install Active Directory, Exchange, SharePoint, ISA/TMG and even SQL on a single box. If that was an acceptable solution, why shouldn’t we do that with just AD and Exchange?

Running Exchange on a domain controller is a supported solution and as long as you understand the impact, go ahead and deploy that single server. When performance, security and flexibility play a role and the budget or infrastructure allows for, it’s recommended to deploy a new Windows server and install Exchange on that server.

How to remove the Exchange Autodiscover SCP

Consider a scenario where you moved all your user’s mailboxes to Exchange Online, for instance after a cutover or staged migration and want to remove any dependencies on the local Exchange server. You may find that Outlook still connects to the local Exchange server for Autodiscover lookups, this is because Outlook is hard-coded to query an AD Service Connection Point to locate a server with the Autodiscover service. When this fails Outlook falls back to the next DNS based methods or uses a local XML file.

Exchange Management Shell

There are multiple ways to prevent Outlook from contacting the local Exchange server first, some of them make more sense than others. The preferred way is to use the Exchange Management Shell to clear the entry for the Client Access server from the SCP.

Set-ClientAccessServer –Identity ServerName -AutoDiscoverServiceInternalUri $null

This removes the SCP entry for this Exchange server.

ADSIEdit

If the above method can no longer be used a low level AD editor as EDSIEdit can be used to remove the SCP manually. The full path of the SCP is:

CN=ServerName,CN=Autodiscover,CN=Protocols,CN=ServerName,CN=Servers,CN=Exchange Administrative Group (FYDIBOHF23SPDLT),CN=Administrative Groups,CN=OrganizationName,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=DomainName,DC=Suffix

This object to remove has the Class type serviceConnectionPoint.

Alternative methods

One of the above steps is basically all you need to do. Alternatives include adding an ExcludeScpLookup value to the \Autodiscover registry key (KB article) and some articles even let you remove the Autodiscover virtual directory from IIS. This will of course make Outlook unable to query the local Exchange server for Autodiscover but why should you if you can simply remove the SCP.

How to verify?

As always, the proof is in the pudding. Use the Outlook test E-mail AutoConfiguration feature to verify the clients behavior. We’re specifically interested in the Log tab where we should see that Outlook is no longer able to query the SCP to obtain the Autodiscover url.

To start the Test E-mail AutoConfiguration tool, follow these steps:

1. Start Outlook.
2. Hold down the Ctrl key, right-click the Outlook icon in the notification area, and then click Test E-mail AutoConfiguration.
3. Verify that the correct email address is in the E-mail Address box.
4. In the Test E-mail AutoConfiguration window, click to clear the Use Guessmart check box and the Secure Guessmart Authentication check box.
5. Click to select the Use AutoDiscover check box, and then click Test.

Earlier I wrote a short article about this tool, unfortunately available in Dutch only: Autodiscover testen met Outlook. But you’ll get the general idea. Focus on the first or third tab when you’re interested in the Autodiscover results, read the Log tab when you're interested in the Autodiscover process.

Exchange 2010 in Extended Support phase, last scheduled UR released today

Exchange 2010 entered the Extended Support phase earlier this year. Basically this means that only security updates will be released to all customers. New and improved features will no longer be added and non-security bugs will be not always be fixed.

Exchange 2010 is a little bit special because it's currently the most widely deployed version of Exchange. And why not, the HA model with the DAG works like a charm, the product is stable and mature and has all the features you can think of. Of course the web interface is looks a bit outdated but it works very well.

The question is if and when customers should upgrade from something that runs pretty well. There is some similarity with Exchange 2003 in the Exchange 2007 and early 2010 timeframe, the software was outdated but it worked very well. This is why so many customers kept running Exchange 2003 until and past the end of Extended Support phase.

Today Microsoft announced Update Rollup 10 for Exchange 2010 SP3. The list with resolved issues counts only 7 bullets.

Very interesting is this little remark at the bottom of the announcement on the Exchange team blog:

Update Rollup 10 is the last scheduled release for Exchange Server 2010. Both Exchange Server 2010 and Exchange Server 2007 are in extended support and will receive security and time zone fixes on-demand on a go-forward basis.

So if you're running 2010, now would be a great time to start planning your upgrade to Exchange 2013, which is rock solid with the 9th CU released today, or Exchange 2016 which is scheduled for release later this year.

Exchange Best practice: Increase the Application log file

Exchange is known as a chatty application when it comes to the Application log. And even while most of the information nowadays is written to the crimson channel, the classic Application still is a great source of information when you need to troubleshoot an issue.

The default settings of the Application log are a maximum size of 16 MB and older events are then overwritten when the log has reached that size. Now this may seem a lot, but can be be only a few hours of information for a busy server in a large environment. Good luck troubleshooting that issue that happened yesterday.

I know some of you are already doing this by default on a new Exchange server deployment, but you should definitely increase the log file size to a larger value. This can be done with the Limit-Eventlog cmdlet.

Limit-EventLog -LogName Application -MaximumSize 100MB

Or on all your Exchange servers at once:

Get-ExchangeServer | % { Limit-EventLog -LogName Application -MaximumSize 100MB -ComputerName $_.name  }

Well, you get the idea. More information: Limit-Eventlog

VMware, let's talk about Exchange and best practices

Apologies for the weird spacing, apparently Google changed something so Windows Live Writer can't post to Blogger currently. Posted this via a very weird work-around.

This week I was made aware of some VMware articles and whitepapers about running Microsoft Exchange on VMware VSAN. VMware VSAN is a 'virtual SAN' solution that employs local server storage and presents this to the hypervisor as a pool of shared storage, similar to the approach of Nutanix.

What surprised me, besides the mud slinging between VMware and their partner Nutanix, is the Exchange environment VMware built to execute their performance testing with.

I won't go into the details of the VSAN storage technology in this blog post, nor cover all aspects of the Exchange architecture design process. Today I want to focus on the high-level Exchange architecture used in the whitepapers.

The Exchange 2010 on VSAN whitepaper

First let's take a look at the 2014 whitepaper Microsoft® Exchange Server Performance on VMware Virtual SAN™.

Wait what? Exchange 2010 was used to perform the tests while Exchange 2013 was released almost two years earlier. Exchange 2010 was installed on Windows Server 2008 R2 while Server 2012 was available. Using older software to demonstrate your platforms capabilities may be done for very good reasons, however they were not explained in the whitepaper.

Microsoft introduced server roles in Exchange 2007 and soon discovered the new roles confused the crap out of their customers. So in the Exchange 2010 time-frame the Exchange team started to emphasize why combining the Mailbox, Client Access and Hub Transport roles on a single server was the preferred way to deploy Exchange. Most important reason: less complexity. And with that of course comes a decrease in costs, better reliability and more good things.

There have been situations where multi-role was not the most efficient option. Back when Exchange 2010 was released the most dominant hypervisor vendor was not able to assign more than 4 vCPU to a VM and when they were, they charged extra for that capability. As far as I know none of these limitations apply today so it's unclear why VMware did not deploy Exchange as recommended.

The Exchange 2013 on VSAN 6.0 whitepaper

The updated version of this whitepaper can be found here: Virtualizing Microsoft Applications on VMware Virtual SAN, Reference Architecture. In this whitepaper VMware explains how to deploy a HA Exchange 2013 environment while using the VSAN solution as the storage back-end.

Now starting with Exchange 2013 the Exchange team went a step further and consolidated their recommendations in the Preferred Architecture. This architecture was presented on industry conferences as MEC, TechEd and recently Ignite. In fact, the first article Microsoft released about Exchange 2016 contains a section called Preferred Architecture where the team explains the Exchange 2013 PA remains valid, with some minor updates.

Now let's compared the PA with the architecture VMware describes in this whitepaper.

First of all VMware did not deploy multi-role servers, they deployed 8 Exchange 2013 server where 4 could've done the trick. More servers equals an increase in complexity and costs.

The placement of the "DAG File Cluster" is a bit confusing too, especially because the Exchange Mailbox servers and the FSW are connected by a blue line. I wonder if the author understands the role of the FSW for the cluster.

Database and log files are isolated, this isn't required. And 80 GB is not sufficient for the OS (32 GB), page file (32 GB + 10 MB) and Exchange (30 GB).

VMware uses two network adapters, which nowadays is no longer the best practice.

Not just VMware, Citrix too

Recently I commented on some Exchange whitepapers by Citrix and today I looked at two whitepapers by VMware. Both vendors demonstrate they have limited understanding of Microsoft Exchange and the best practices Microsoft wrote.

Let me be honest. IT has been never more challenging as today. Customer's environments are growing in size and complexity. Trends like virtualization (server, network, storage, application, etcetera), BYOD, consumerization of IT, cloud computing and increasing demands from the business can drive an IT person crazy. This is why it has never been so important for vendors to reduce complexity, make sure recommendations are in line with other vendors and that customers receive all the help they need to implement the solutions in the best way possible.

As an Exchange consultant I often sat with customers discussing Microsoft best practices for Exchange and the storage or virtualization guy showing up with an outdated or slightly inaccurate document instructing the exact opposite. Customers expect the vendors to help them clarify stuff, not to cause even more confusion.

So Citrix, VMware and other vendors too, time to step up your game. Make sure you read and understand every article on the official Exchange Team Blog. Update your whitepapers during their lifetime when progressive insight or updated best practices require it. Ask Microsoft for feedback or involve Subject Matter Experts to review your document and provide some feedback from their perspective. I'm sure that with a little extra effort the quality of the whitepapers can improve a lot.

PAL tool now works with Exchange 2013!

PAL or Performance Analyzer for Logs is a handy tool to assist with performance monitoring and troubleshooting. Many Exchange admins have used this tool to quickly create a performance counter set and scan the results against a threshold file. PAL outputs a report with easy graphs and performance alerts. This report is a great help to get a quick overview of the performance of a server and indication of possible performance bottlenecks.

Unfortunately there was no threshold file for Exchange 2013, partly because Microsoft never published the detailed performance counter information as they did for 2007 and 2010. I am very excited that Adrian Moore (admoore@microsoft.com) found the time to write the threshold file for Exchange 2013, which is included with PAL version 2.7.3.

Download PAL and read more at the PAL site on Codeplex.

Allow or disallow Outlook Online Mode per user

Outlook Cached Mode offloads IOPs from the server to the workstation, provides offline access and also makes the user experience less vulnerable to unreliable or high-latency internet connections. So Cached Mode is recommended for Exchange Online and is the the preferred mode to connect Outlook with Exchange.

Unfortunately there are some scenario's where you can't enable Cached Mode, for instance stateless (VDI) desktops or RDS/TS hosts when customers for some reason are not able to store the .OST file on a fileserver.

So far nothing new for you as an experience Exchange admin. But did you know that you can specify per-mailbox if it's allowed to access with Outlook in Online Mode? This can be controlled with the Set-CASMailbox cmdlet. This cmdlet is available in both Exchange 2010 and 2013, however the switch to allow Online Mode is not available in Exchange Online.

It's very easy to block access with Outlook in Online Mode for a single mailbox:

Get-Mailbox <user> | Set-CASMailbox -MAPIBlockOutlookNonCachedMode:$true

A value of True enables the blocking of Online Mode clients, set the value to False to disable checking of the connection mode. False is the default value.

The user experience is not that great, a rather generic error message is displayed when the user tries to connect in Online Mode:

So if you have the requirement to block Outlook in Online Mode, you can do this with the Set-CASMailbox and the MAPIBlockOutlookNonCachedMode switch.

Beware though, setting MAPIBlockOutlookNonCachedMode to $true breaks New-MailboxExportRequest which means you can't export the content from this mailboxes to PST. Not something you would do every day, but good to know.

Exchange 2010 SP3: UCMA requires the following missing Windows features

When installing Exchange 2010 SP3 on a server with Windows Server 2012 you need to install some prerequisites first. Each server with the Client Access role requires the Unified Communications Managed API 4.0 Runtime.

You may run into this error message when you try to install:

Microsoft Unified Communications Managed API 4.0, Runtime requires the following missing Windows features.

• Media Foundation

This error occurs when you followed the instructions on this page: Exchange 2010 Prerequisites (Install the Windows Server 2012 operating system prerequisites). For a server with the three basic roles Microsoft says you need to install the following operating system components:

Unfortunately Server-Media-Foundation is missing, that's why the UCMA installation fails. So we need to add this one too:

Add-WindowsFeature Server-Media-Foundation

And while we're on it, don't forget to add Telnet-Client because this basic troubleshooting tool should be installed on every server with Exchange by default.

Exchange and support for CNG/KSP certificates

Cryptography Next Generation (CNG) is a set of APIs which can be found in Windows Server 2008 and newer. CNG offers an advanced set of features to create hashes, encrypt and decrypt data and to manage keys and cryptographic providers. CNG implements the United States government's Suite B cryptographic algorithms. As the hipsters say, CNG is the new PKI but you probably haven't heard of it.

One of the features of CNG is to create certificates that that use a Key Storage Provider (KSP) to store the private key, as opposed to a Cryptographic Service Provider (CSP) like regular certificates do. Exchange does not support these types of certificates for securing OWA and ECP. You will notice this immediately because your users will return back in the FBA screen after logging in after you installed such a certificate.

Read the following article for more information and workarounds: Outlook Web App and ECP redirect to the FBA page in Exchange Server 2013

Outlook Anywhere Kerberos and moving to Exchange 2013? Read this first.

If you run Exchange 2010 and are using Kerberos authentication for Outlook in a load-balanced environment you probably have scheduled the RollAlternateserviceAccountPassword.ps1 script. This script updates the alternate service account credential (ASA credential) and pushes the new value to your CAS servers. Common parameter options are -ToEntireForest, -ToArrayMembers or -CopyFrom with -ToSpecificServers. If you are familiar with the script, I assume they need no clarification.

Now consider a scenario where you used -ToArrayMembers and you add your first Exchange 2013 CAS server to that site. The script uses the Get-ClientAccessArray cmdlet to query the members of this array, this cmdlet returns the Exchange 2013 CAS servers in that site too.

Unfortunately the script is not able to update the ASA credential on both 2010 and 2013 servers. This causes the script to fail when it tries to process the Exchange 2013 CAS server and ultimately cancels the process of updating and synchronizing the ASA credential. Unless you configured specific monitoring for this process you'll probably won't notice the issue before stuff breaks and users start complaining. To check if this issue applies to your environment:

Get-ClientAccessServer -IncludeAlternateServiceAccountCredentialstatus |Fl Name, AlternateServiceAccountConfiguration

By the way, the RollAlternateserviceAccountPassword.ps1 script writes a log file in the $exinstall\Logging\RollAlternateServiceAccountPassword folder. Be ware this log is written on the server where the script is executed, this is not necessarily the server where you scheduled the script to run.

There are several workarounds to prevent this from happening. First you could consider to deploy the Exchange 2013 CAS servers in another site. Another option is to update a single server first and then use the -CopyFrom and -ToSpecificServers switches to update your Exchange 2010 CAS servers which you have to specify.

For more information on planning the migration from Exchange 2010 to Exchange 2013 with regards to Kerberos authentication I recommend this excellent article on the Exchange Team Blog: Exchange 2013 and Exchange 2010 Coexistence with Kerberos Authentication

Windows Mobile does not support your new SSL certificate

The world is moving away from SHA-1 certificates, which is a good thing from a security perspective. Major vendors like Microsoft and Google are forcing Certificate Authorities to give out certificates with the new SHA-256 hashing algorithm. They do this by simply stop trusting root CA’s which give out SHA-1 certificates after a certain date.

What can you do? Not much actually, because every new SSL certificate you purchase or renew will be SHA-256. Of course there are old operating systems which don't support SHA-256, for instance XP versions older than SP3 and Server 2003 without SP2 installed. So basically every modern OS and even much older versions support SHA-256.

Now for us Exchange guys there may be an exception, this is Windows Mobile 5 and 6 which do not support SHA-256. Yes, this operating system is very old and has been superseded by Windows Phone 7 and newer. However, it has been sold until not very long ago and products with Windows Mobile may be still available. An example of such a device is the Samsung Omnia 735.

So if you have any users with a Windows Mobile device and are planning to renew your Exchange SSL certificate, please be aware of this issue. Depending of your situation you may consider replacing the devices or even move to SHA-1 certificates which are signed by your own CA. This obviously is not a safe long-term solution but it may buy you some time before you can move to SHA-256 certificates.

Autodiscover testen met Outlook

Outlook die zijn eigen profiel configureert zonder dat de gebruiker hier iets aan hoeft te doen, en het werkt ook voor ActiveSync! Het kan sinds Exchange 2007 en Outlook 2007, maar Autodiscover heeft van het begin af voor verwarring gezorgd. In dit artikel ga ik niet in over de exacte werking maar laat ik een 'client-side' troubleshooting tool zien die ons zowel de werking van het proces laat zien, als de daadwerkelijke informatie die de client van de Autodiscover Service krijgt.

Je hebt nodig:

• Outlook 2007 of hoger
• Een Outlook profiel

Dat Outlook profiel mag een leeg profiel zijn of eentje met nepgegevens, bijvoorbeeld voor POP3 en SMTP.

Zoek het Outlook icoontje in de System Tray, klik er met de rechtermuisknop op terwijl je CRTL ingedrukt houdt. Kies nu Test E-mail AutoConfiguration:

Wanneer je als AD gebruiker op een AD member computer ingelogd bent leest Outlook het windowsEmailAddress attribuut uit en vult deze alvast in. In alle andere gevallen vul je je mailadres en wachtwoord in. Haal de vinkjes voor Guessmart weg.

Klik nu op Test, waarna Outlook een Autodiscover lookup gaat doen met de hierboven ingevulde gegevens. Afhankelijk van de configuratie kan dit even duren, het is dan ook interessant om tijdens het uitvoeren direct het tabblad Log in de gaten te houden, hier kunnen we de voortgang van het proces volgen:

In dit geval gaat het om een Office 365 mailbox waarbij het proces wat complexer is, maar het belangrijkste is dat de laatste regels aangeven dat het proces geslaagd is. Bij een ander resultaat betekent dit dat je de stappen doorleest en na gaat of ze overeenkomen met de wijze waarop je Autodiscover geconfigureerd hebt.

Dan terug naar het tabblad Results, deze geeft het ontvangen resultaat netjes weer maar bevat feitelijk dezelfde informatie als de ruwe XML die we op het derde tabblad kunnen zien.

Op deze plek controleren we de daadwerkelijk ontvangen informatie, grotendeels bepaald door wat we als InternalURL en ExternalURL waardes hebben geconfigureerd in Exchange.

Met dit handige hulpmiddel wordt het troubleshooten van Autodiscover een stuk eenvoudiger.

Spamhaus RBL werkt niet met Google DNS

In mijn Exchange 2013 lab heb ik een Edge Transport server geconfigureerd om de Spamhaus Real-time Block List te gebruiken. Door een IP Block List Provider toe te voegen en de Connection Filtering agent in te schakelen test Exchange of de mailserver die een bericht wil afleveren niet op een RBL staat.

Helaas bleek dat bij mij niet te werken, dit werd bevestigd door een test met behulp van de Crynwr Spamhaus test. Raar, want het is een eenvoudige configuratie:

Add-IPBlockListProvider -Name Spamhaus -LookupDomain zen.spamhaus.org

De Connection Filtering transport agent staat standaard ingeschakeld, dat was het probleem dus ook niet. De volgende stap is het handmatig controleren of je een DNS query naar de servers van Spamhaus kunt doen. Dit kun je doen door een IP-adres te kiezen, bijvoorbeeld 127.0.0.2, en hiermee een query te construeren. Zet het IP-adres in omgekeerde volgorde en laat deze eindigen op .zen.spamhaus.org. In dit geval moeten we dus een DNS query doen voor 2.0.0.127.zen.spamhaus.org, het antwoord bevat een code die aangeeft of het IP wel of niet op de blocklist staat. En in dit geval zijn we niet geïnteresseerd in het antwoord maar willen we in eerste instantie controleren of de query überhaupt mogelijk is.

nslookup 2.0.0.127.zen.spamhaus.org

In dit voorbeeld gebruik ik een interne DNS-server waarvan ik de naam verborgen heb, maar in mijn lab mislukte deze test. De oorzaak bleek te liggen in de publieke Google DNS servers die ik als forwarder had geconfigureerd, dit zijn de populaire 8.8.8.8 en 8.8.4.4 adressen. Wanneer we dezelfde query uitvoeren tegen één van deze DNS-servers dan mislukt deze:

nslookup 2.0.0.127.zen.spamhaus.org 8.8.8.8

Bij nader onderzoek blijkt dat Spamhaus dit ook vermeldt in hun FAQ:

Check what DNS resolvers you are using: If you are using a free "open DNS resolver" service such as the Google Public DNS or large cloud/outsourced public DNS servers, such as Level3's or Verizon's, to resolve your DNSBL requests, in most cases you will receive a "not listed" (NXDOMAIN) reply from Spamhaus' public DNSBL servers. We recommend using your own DNS servers when doing DNSBL queries to Spamhaus.

In andere gevallen kan er natuurlijk een andere oorzaak zijn, maar deze troubelshooting methode kan een eerste stap zijn om te controleren of je de servers van Spamhaus kunt raadplegen.

Bron

Google Chrome 37 en Exchange: 9 vragen en antwoorden

Gisteren schreef ik over een paar features in Exchange 2013 EAC die niet meer werken in Chrome 37. Inmiddels zie ik ook berichten van mensen waarbij bepaalde knoppen in Exchange 2010 OWA niet meer werken, dit blijkt te herleiden naar dezelfde oorzaak. In dit artikel leg ik uit wat het probleem is, schets de context en leg uit of we hier wat aan kunnen doen.

Wat is het issue dan?

Microsoft maakt in OWA en ECP/EAC gebruik van de showModalDialog method, deze methode wordt gebruikt om een dialog box te openen met daarin een specifieke HTML-pagina. Voorbeelden van plaatsen waar deze methode gebruikt wordt zijn de From: en To: buttons in OWA 2010 en het + en potlood icoon in Exchange 2013 EAC. Het volgende scherm is een voorbeeld van een dialog box die op deze manier wordt aangeroepen:

In Chrome 37 werkt de showModalDialog method niet meer en verschijnen deze dialog boxes dus niet. Overigens raakt dit niet alleen Microsoft applicaties zoals Exchange, SharePoint en Dynamics maar ook verschillende producten van andere leveranciers en een grote hoeveelheid (interne) websites die van deze methode gebruik maken.

Waarom werkt dit niet?

De ontwikkelaars van Google Chrome hebben er voor gekozen om showModalDialog standaard uit te schakelen in Chrome 37. In een volgende release zal showModalDialog compleet verwijderd worden.

Zou het moeten werken?

Als je het aan de ontwikkelaars van Chrome vraagt dan wordt uitgelegd dat deze feature ooit in IE4 zat en sindsdien is bijgehouden in andere browsers maar eigenlijk nooit had mogen bestaan. Als je het aan Microsoft vraagt dan zullen die antwoorden dat ze Chrome 37 ondersteunen als client voor OWA in de premium versie:

Waarom is deze keuze gemaakt?

Volgens de ontwikkelaars van Blink, de rendering engine van Chromium waar ook Chrome op gebaseerd is, kost het bijhouden van deze methode erg veel moeite, de code is erg complex geworden en er zijn afhankelijkheden van andere twijfelachtige code. Verder is een kenmerk van deze methode dat hij a) een pop-up geeft, wat niet goed werkt op mobiele platformen en b) de rest van de browser bevriest zolang de pop-up actief is. Dit heeft zelfs al tot een aantal security vulnerabilities geleid.

Als deze methode maar weinig gebruikt wordt en uiteindelijk geheel uit de code verwijderd kan worden dan zou dat veel voordeel brengen voor de programmeurs.

Sinds wanneer?

Aangezien het lijkt dat de hele wereld, inclusief Microsoft, verrast is door deze aanpassing is het goed om eens naar de tijdlijn te kijken. In oktober 2013 is onder Chromium ontwikkelaars voorgesteld om een teller toe te voegen om het gebruik van showModalDialog te meten en een waarschuwing te geven aan website-ontwikkelaars. Hiermee wordt showModalDialog 'deprecated' verklaard, ofwel een feature die uitgefaseerd gaat worden. Op 1 oktober 2013 is dit voorstel verwerkt in de code en op 10 april 2010 beland in de Chrome 35 Beta. Op hetzelfde moment is aangekondigd om showModalDialog in Chrome 36 geheel te verwijderen.

Door het toevoegen van de teller werd vastgesteld dat slechts 0,006% van de pageviews deze methode gebruikt:

Voor de ontwikkelaars werd dit gezien als bevestiging dat bijna niemand dit gebruikt. In de hierop volgende discussie kwam er wel wat tegengas maar werd er vooral met het voorstel ingestemd. Ik betwijfel of iemand zich realiseert hoe groot de impact is om de functionaliteit van 0,006% van de pageviews stuk te maken, heeft iemand dat percentage omgerekend naar absoute aantallen? Is er een inventarisatie gemaakt van bekende applicaties die in deze categorie vallen om de impact beter te kunnen begrijpen?

De definitieve versie van Chrome 35 verscheen op 20 mei van dit jaar, Chrome 36 volgde al op 16 juli en Chrome 37 een maand later, op 26 augustus. In tegenstelling tot wat eerder aangekondigd was is showModalDialog in Chrome 36 nog steeds aanwezig maar in 37 is deze uitgeschakeld. In verschillende blogposts is de keuze om showModalDialog te verwijderen verder toegelicht en beargumenteerd.

Uitgeschakeld of verwijderd?

In Chrome 37 is de feature uitgeschakeld maar weer in te schakelen tot 15 mei 2015, daarna wordt de feature echt verwijderd. En hier wordt het interessant. Chrome kent een policymodel waarmee bepaalde features geconfigureerd kunnen worden, bedoeld om Chrome geschikt te maken voor de zakelijke markt. We kunnen de policy EnableDeprecatedWebPlatformFeatures gebruiken om een uitgefaseerde maar nog niet verwijderde features weer aan te zetten.

Hoe zet ik het weer aan?

Met (een elevated instance van) PowerShell zouden we dit als volgt doen:

New-Item HKLM:\SOFTWARE\Policies\Google\Chrome\EnableDeprecatedWebPlatformFeatures
Set-ItemProperty HKLM:\SOFTWARE\Policies\Google\Chrome\EnableDeprecatedWebPlatformFeatures 1 ShowModalDialog_EffectiveUntil20150430 -type string

Bij het testen van deze code bleek HKLM:\SOFTWARE\Policies\Google\Chrome op mijn computer niet te bestaan:

En ook na het handmatig toevoegen van de hyve bleek de instelling niet doorgevoerd te worden in Chrome, zo bevestigd een bezoek aan chrome://policy/.

Om te begrijpen waarom moeten we weer even terug naar het ontwikkelproces van Chrome. De policies konden nu geconfigureerd worden door middel van registry keys of een GPO policy template. Nadat Chrome registry keys ging gebruiken om de configuratie te managen bleek al snel dat deze registry settings ook aangepast konden worden door andere programma's, denk bijvoorbeeld aan malware. Als reactie ging Chrome 28 de registry keys negeren en kon alleen nog het GPO template gebruikt worden om Chrome te configureren. Wat uiteraard tot veel klachten leidde van beheerders die geen AD gebruiken, geen GPO's of beheer standaard op basis van registry keys uitvoeren.

De verandering in Chrome 28 werden nu deels teruggedraaid en vanaf Chrome 35 worden de registry keys ook weer gebruikt, alleen als de computer lid is van een Active Directory domein.

Wat nu dan?

Beheerders van werkplekken in een AD domein kunnen de registry key of het GPO template gebruiken. Voor losstaande werkplekken kan de GPO template ook in de Loacal Group Policy (gpedit.msc) gelezen worden:

1. Download de policy templates hier.
2. Open gpedit.msc en ga naar "Computer Configuration" -> rechtsklik "Administrative Templates" -> "Add/Remove templates" -> klik "Add..." -> selecteer "windows/adm/en-US/chrome.adm" uit de policy_templates.zip download
3. In "Computer Configuration" -> "Administrative Templates" -> "Classic Administrative Templates" -> "Google" -> "Google Chrome" configureer je nu de policy settings.
4. Open nu Chrome en ga naar chrome://policy/, hier moet de aangepaste policy nu zichtbaar zijn.

Ontwikkelaars van website zullen alle instances van showModalDialog moeten vervangen door een alternatief.

Wat vinden we hier van?

Het hele verhaal roept de vraag op in hoeverre Chrome klaar is voor de zakelijke markt. De razendsnelle ontwikkelingscyclus van dit open-source project verrast beheerders en ontwikkelaars van websites door features uit te schakelen. Hetzelfde geldt voor het uitschakelen van het beheer door middel van registry keys, zoiets kun je niet doen zonder de zakelijke markt zorgvuldig te informeren en een alternatief te bieden.

Maar ook de afwezigheid van Microsoft in deze discussie valt op. Bij mijn onderzoek viel me op dat er uitgebreide discussies plaatsvinden en hebben gevonden binnen de Chromium, Chrome en Opera communities maar ook bij Mozilla Firefox waar de zelfde discussie speelt. Is er dan geen enkele developer bij Microsoft geweest die zich realiseerde dat zijn applicatie volop gebruik maakt van deze feature? En dat juist nu Microsoft OWA duidelijk positioneert als de primaire client om te werken met e-mail.

Zakelijke klanten zullen zich afhankelijk van hun contract tot Premier Support richten of een case openen via de reguliere supportkanalen. Gebruiker van Office 365, ofwel hun beheerders, kunnen een support request openen In het Office 365 admin center. Ik verwacht een bericht op het Exchange Team Blog één dezer dagen en volg de discussies in de communities met veel interesse.

Werken met (legacy) Public Folders

Op dit moment heb ik het genoegen om te mogen werken met een grote hoeveelheid Public Folder data. Het doel is om deze te repliceren naar nieuwe databases. Hierbij gebruik ik een aantal scripts en tools die misschien handig zijn om te delen.

Overzicht

Als eerste heb ik een Excel-file gemaakt met een overzicht van alle folders, het aantal items hierin en de grootte in MB. Dit is weliswaar een momentopname maar het geeft je een goed beeld van de structuur en de grootste folders.

Get-PublicFolder -Recurse -ResultSize Unlimited | Get-PublicFolderStatistics | Select-Object Name,FolderPath,CreationTime,LastAccessTime,LastModificationTime,LastUserModificationTime,LastUserAccessTime,@{expression={$_.totalitemsize.value.ToMB()};label=”Size(MB)”},ItemCount | Export-CSV "C:\data\pfstats.csv"

Denk aan de -ResultSize parameter, anders krijg je met Get-PublicFolder alleen de eerste 10.000 resultaten. Dit bestand heb ik in Excel geopend, conditional formatting gebruikt om folders met veel items of data snel te kunnen herkennen en uiteraard opgeslagen als xlsx-bestand.

Aan de hand van dit bestand heb ik de top-level folder kunnen kiezen waarmee ik de replicatie wil starten. Ook heb ik kolommen toegevoegd voor de nieuwe databases zodat ik een check kan zetten als ik deze als replica toegevoegd heb. Dit helpt mij om overzicht te houden gedurende het proces.

Replica's toevoegen of verwijderen

Op de locatie waar Exchange geïnstalleerd is staat een \scripts directory. Omdat het installatiepad per server verschilt is er de $exscripts een variabele die we in Exchange Management Shell kunnen gebruiken om de scripts te lokaliseren.

In dit geval gebruik ik het AddReplicaToPFRecursive.ps1 script om de nieuwe database als replica toe te voegen op een toplevel folder en alle onderliggende folders:

cd $exscripts
./AddReplicaToPFRecursive.ps1 -TopPublicFolder "\MyFolder" -ServerToAdd NewServer1

Op een later moment gebruik ik het RemoveReplicaFromPFRecursive.ps1 om de oude replica te verwijderen. Een alternatief zou het gebruik van het MoveAllReplicas.ps1 script die beide stappen in één keer doet. Meer informatie over deze scripts vind je hier: Scripts for Managing Public Folders in the Exchange Management Shell

Rapportage en monitoring

Om op hoofdlijnen te zien op de replicatie plaats vindt, gebruik ik het volgende commando die de grootte van het .edb-bestand geeft:

Get-PublicFolderDatabase | Select Server, Name, @{Name="Size (GB)";Expression={$objitem = (Get-PublicFolderDatabase $_.Identity); $path = "`\`\" + $objitem.server + "`\" + $objItem.EdbFilePath.DriveName.Remove(1).ToString() + "$"+ $objItem.EdbFilePath.PathName.Remove(0,2); $size = ((Get-ChildItem $path).length)/1048576KB; [math]::round($size, 2)}}, @{Name="Size (MB)";Expression={$objitem = (Get-PublicFolderDatabase $_.Identity); $path = "`\`\" + $objitem.server + "`\" + $objItem.EdbFilePath.DriveName.Remove(1).ToString() + "$"+ $objItem.EdbFilePath.PathName.Remove(0,2); $size = ((Get-ChildItem $path).length)/1024KB; [math]::round($size, 2)}}

Deze one-liner ziet er misschien complexer uit dan hij is, dat komt omdat we het UNC pad moeten samenstellen om de grootte van het bestand te kunnen meten. Een andere stukje code is om de grootte om te rekenen naar GB en MB.

Door dit commando af en toe te herhalen zie je al snel dat het aantal GB's van de nieuwe databases toeneemt. Dit is natuurlijk niet voldoende om een goed beeld van de status te krijgen. Dus na enige tijd heb ik het Exchange 2010 Public Folder Replication Report van Mike Walker gebruikt.

Dit script gebruikt Get-PublicFolder en Get-PublicFolderSatistics om informatie over de public folders te vergaren. Omdat dit zeer lang kan duren, al snel enkele uren bij een paar duizend folders, beperkt ik het rapport tot een specifieke top-level folder:

.\Get-PublicFolderReplicationReport.ps1 -FolderPath "\MyFolder" -Recurse -ComputerName @("OldServer1", "OldServer2", "NewServer1", "NewServer2") -AsHTML -Filename report.html

Het resultaat is een zeer bruikbaar rapport in HTML-vorm.

Troubleshooting

Op basis van de bestandsgrootte was al duidelijk dat één van de nieuwe databases niet volledig gevuld werd en het HTML-rapport bevestigt dat. Om uit te vinden of er een probleem is heb ik het logging level voor een aantal relevante bronnen op de bron- en doelserver aangepast van Lowest naar High:

Set-eventloglevel -Identity "OldServer1\MSExchangeIS\9001 Public\Replication Incoming Messages" -level high
Set-eventloglevel -Identity "OldServer1\MSExchangeIS\9001 Public\Replication Outgoing Messages" -level high
Set-eventloglevel -Identity "OldServer1\MSExchangeIS\9001 Public\Replication NDRs" -level high
Set-eventloglevel -Identity "OldServer1\MSExchangeIS\9001 Public\Replication Backfill" -level high
Set-eventloglevel -Identity "OldServer1\MSExchangeIS\9001 Public\Replication Errors" -level high
Set-eventloglevel -Identity "OldServer1\MSExchangeIS\9001 Public\Replication General" -level high
Set-eventloglevel -Identity "NewServer1\MSExchangeIS\9001 Public\Replication Incoming Messages" -level high
Set-eventloglevel -Identity "NewServer1\MSExchangeIS\9001 Public\Replication Outgoing Messages" -level high
Set-eventloglevel -Identity "NewServer1\MSExchangeIS\9001 Public\Replication NDRs" -level high
Set-eventloglevel -Identity "NewServer1\MSExchangeIS\9001 Public\Replication Backfill" -level high
Set-eventloglevel -Identity "NewServer1\MSExchangeIS\9001 Public\Replication Errors" -level high
Set-eventloglevel -Identity "NewServer1\MSExchangeIS\9001 Public\Replication General" -level high

Na enige tijd verschijnen er in het Application eventlog allerlei meldingen van source MSExchangeIS Public Store. Op zich zou ik die met PowerShell kunnen analyseren (Get-EventLog) maar voor een snel overzicht heb ik de Event Viewer gemaakt en een filter op source MSExchangeIS Public Store gezet.

De grote hoeveelheid errors bleken vals alarm te zijn, deze waren onschuldig en geen indicatie voor een daadwerkelijk probleem.

Om eventuele fouten te repliceren heb ik de replicatie voor een (op de nieuwe server niet gevulde) folder handmatig gestart:

Update-PublicFolder "MyFolder\SubFolder" -Server OldServer1

Op dit moment zie je direct de bijbehorende replicatie-events verschijnen op de bron- en doelserver. Vervolgende de statistieken voor deze folder opgevraagd op de oude en nieuwe server:

Get-PublicFolderStatistics "MyFolder\SubFolder" -Server OldServer1
Get-PublicFolderStatistics "MyFolder\SubFolder" -Server NewServer1

Na een paar minuten gaf de folder op de nieuwe server het zelfde aantal items als de oude. Daarom ook de rest van de folders nogmaals laten updaten:

Get-PublicFolder "MyFolder\SubFolder" -Recurse -ResultSize Unlimited | Update-PublicFolder -Server OldServer1

Daarna het Public Folder Replication Report nogmaals uitgevoerd en uitsluitend groene vlakjes gezien.

Tips

Met name bij het werken met grote hoeveelheden folders duren eenvoudige taken als het opvragen van folders of statistieken zeer lang. Plan dus vooruit en voorkom dat je langlopende taken opnieuw moet uitvoeren. Sla het resultaat van een query op in een CSV-bestand of variabele.

Deel de hoeveelheid data op in behapbare blokken, bijvoorbeeld een toplevelfolder met onderliggende mappen die voldoende items bevatten maar pakweg een paar GB groot zijn. Selecteer hiervoor desnoods een dieper gelegen folderstructuur. Besteed aandacht aan het verloop en zorg dat je goed inzicht hebt in het verloop of eventuele problemen. Los die eerst op en ga dan verder met de rest van de data.

Dit artikel heeft betrekking op Exchange 2010 maar is ook bruikbaar voor Exchange 2007. Om Public Folder replicatie goed te begrijpen moeten we terugvallen op documentatie voor Exchange 2003, afgezien van de gebruikte tools is er in Exchange 2007 en 2010 namelijk weinig veranderd. Een goed stuk documentatie is: Controlling Exchange Server 2003 Public Folder Replication