Friday, November 4, 2016

Friendly reminder: /RecoverServer to an older OS version is not supported

In case you missed it, it’s currently not recommended to deploy Exchange 2016 on Windows Server 2016. The Windows Server team added a section dedicated to Exchange in their Windows Server 2016 Release Notes:

If you attempt to run Microsoft Exchange 2016 CU3 on Windows Server 2016, you will experience errors in the IIS host process W3WP.exe. There is no workaround at this time. You should postpone deployment of Exchange 2016 CU3 on Windows Server 2016 until a supported fix is available.

image

It’s expected that a blog post with a similar message will appear soon on the Exchange Team Blog. The difference with the first post is that this post has to include recommendations to customers that deployed Exchange 2016 on Windows Server 2016, which is supported since CU3. As the Windows Server team already stated, there is no fix or workaround available today.

To make matters worse, it looks like this issue only applies to Server 2016 servers that have been added to a DAG. Good news for customers with stand-alone Exchange 2016 on Windows Server 2016 servers but it makes recovery a tad bit more complicated.

The supported approach would be something like this:

  1. Stand up new Windows Server 2012 R2 servers
  2. Install Exchange 2016 CU3 (or CU2 to prevent this from happening)
  3. Optional: Add both servers to a DAG (not the same one, 2012 R2 and 2016 servers can’t be members of the same DAG)
  4. Move the mailboxes to the new servers
  5. Uninstall Exchange 2016 from the impacted servers

The big question here is, will the admin be able to perform all these tasks with Exchange 2016 being in this state: Exchange Server 2016 & Windows Server 2016 Datacenter IIS AppPool's constantly crashing. ECP/Powershell inaccessible

A user in a community recommended this approach:

image

There is no fix at the moment. Therefore if you have put Exchange 2016 on to Windows 2016 then the only option is to shutdown the server, rebuild as Windows 2012 R2 then do a recover server install of Exchange 2016. That works and allows you to remove it cleanly.

Unfortunately this process is not supported (*) and never has been. This can be found in Recover an Exchange Server:

What do you need to know before you begin?
The server on which recovery is being performed must be running the same operating system as the lost server. For example, you can't recover a server that was running Exchange 2013 and Windows Server 2008 R2 on a server running Windows Server 2012, or vice versa. Likewise, you can’t recover a server that was running Exchange 2013 and Windows Server 2012 on a server running Windows Server 2012 R2, or vice versa.

So deploying a new server with Server 2012 R2 and then install Exchange 2016 with the /RecoverServer option is not supported.

Let’s see what the Exchange team will have to say on this.

(*) Not supported in this context means that Microsoft Support cannot support your environment if you do this. The exception is if Microsoft Support asks you to follow this process, in that case you will be supported.

Microsoft Teams, here’s the admin guidance

Yesterday Microsoft released a preview version of Microsoft Teams, the new product to compete with Slack. It will be very interesting to see Teams integrate in Office 365 as yet another collaboration product next to Skype for Business, Exchange, Yammer, SharePoint Newsfeed and Office 365 Groups.

Currently Microsoft Teams is not enabled by default, a tenant admin has to enable the application in the Office 365 Admin Center.

image

This will change in 2017 Q1 when Teams will be enabled by default. Or as Microsoft states in the Message Center (id MC84541):

We are rolling this preview out off-by-default, to give you a chance to explore the new capabilities. This experience will be turned on-by-default in the first quarter of 2017. At this time, you can control access to this experience, at the organization level. We are working on user-level controls for this feature, and will communicate again when available.

I know that many of my customers are still struggling how to handle Office 365 Groups creation and the lack of governance and control around this area. Microsoft is working hard to improve this, as they explained in this session at Ignite 2016: Manage Microsoft Office 365 Groups But with every new Microsoft Team there will be a corresponding Office 365 Group created.

This raises the question how admins can manage the Microsoft Teams roll-out in their organization. What are the firewall requirements? And how to estimate bandwidth for peer-to-peer calling?

The good news is, there is actual admin documentation. The bad news is that it’s no longer consolidated on a single place as we’re used to with TechNet. The currently available resources for admins can be found here:

Much to my surprise the most relevant information for admins was hidden in part 2 of the MVA video series. I highly recommend to download the Deploy_Microsoft_Teams.pdf document that can be found in the Resources section of this training.

image

There’s a ton of extremely valuable information in this document that helps you understand how to prepare for Microsoft Teams, at least from an infrastructure point-of-view.

image

image

image

Have fun!

Office 365 MFA is awesome! Unless you’re an administrator…

For some reason I have never worked with MFA with Office 365 until last year. And I must say, it is awesome! Even the free version of Azure MFA that’s included with the Office 365 subscription meets the requirements of most organizations. It’s very easy to setup and configure and the end-user experience is pretty good too, supporting text messages, phone call or the Azure Authenticator app.

image

Microsoft did a great job integrating the PhoneFactor acquisition (2012) in Azure AD and Office 365. So it’s not a surprise that a lot of organizations plan to enable MFA for all users, some users or the users with an admin role. And that’s where the issue is, Office 365 MFA currently does not support Remote PowerShell. Or I should say Remote PowerShell does not offer support for MFA because this would require support for Modern Authentication. This applies not only to Exchange management, but too PowerShell management of SharePoint, Skype for Business, EOP and Security & Compliance as well.

image

How about app passwords then? We can use app passwords for applications that do not support MFA right? Unfortunately app passwords are not working either.

When talking with Microsoft Premier Support they explained there’s currently no news to share. However, a Microsoft Most Valuable Professional explored the limits of his NDA on Facebook when he disclosed that Microsoft has made a preview version of Exchange PowerShell to beta testers at the moment. I’m very keen to learn more about this new version, because currently Remote PowerShell depends on the version of PowerShell that’s installed in the OS of the workstation. I’m assuming that MFA support requires the installation of additional software.

Ironically the new Office 365 Secure Score site (https://securescore.office.com/), a challenge were organizations receive points for increasing the security, awards 50 points for organizations that enable MFA for all their Tenant Admins. There’s no mention that this removes the ability to manage Office 365 with PowerShell.

image

Keep an eye on the Office 365 Roadmap and the Azure MFA Documentation for updates.

Microsoft reverses the undocumented changed Exchange Online license removal behavior

A couple of weeks ago some users reported a change in behavior when an Exchange Online license was removed from an Azure AD user object. It was reported that with the changed behavior mail would still flow to a mailbox, after the license was removed. When a customer opened a service request with Microsoft Support, they acknowledged the change but there was no documentation available, nor was this change announced on the Office 365 Roadmap.

The first public information appeared early October when Microsoft added the following text in the License Removal section of the Delete or restore user mailboxes in Exchange Online document.

Previously, in Exchange Online, if you removed an EXO license the user was kept, but the mailbox user was transformed into a mail user and the mailbox was moved to the recycle bin. You could then recover the mailbox within the 30 days time limit.

This has now changed in Exchange Online. If you remove a user's license, the user mailbox will no longer be able to sign in and use Exchange Online or Office 365. The user mailbox will remain in Exchange Online until it is deleted, permanently removed or purged by the Office 365 admin. You can reassign a license to the user and make the mailbox active again.

But if you look in this document today the text was removed and replaced with a link to this post on the Official Exchange Team Blog: Change in behavior for delicensed Exchange Online users

image

In this post Microsoft explains the change in more detail, apologizes for the way they handled the change and most important, that they rolled back the change for now:

Due to extensive feedback from customers we are rolling back this feature to the original behavior in order to improve the feature before we release it again in an easier format (and with better documentation).

For more information, read the full post here: Change in behavior for delicensed Exchange Online users

Tuesday, October 18, 2016

Microsoft and their frequent product name changes

Today I learned that the Microsoft Federation Gateway was rebranded to Azure Authentication System some time ago. This reminded me of that one time when we made a marketing video and had the narrator use the name Windows Azure when Microsoft decided to rebrand the service to Microsoft Azure. We had to pay the voice actor again to record the text for a second time, now with Microsoft Azure.

Word Cloud-1Sometimes a new name makes perfect sense, for instance when Microsoft renamed the RPC over HTTP protocol to Outlook Anywhere. Or when they rebranded Business Productivity Online Suite (deskless worker anyone?) to Office 365.

In the Exchange space the favorite buzzword today seems to be Modern. The newly architected Public Folders in Exchange 2013 and Exchange Online became Modern Public Folders. The smoother ADAL based authentication became Modern Authentication and I overheard someone jokingly using the term Modern Outlook Anywhere for MAPI/HTTP.

Smart branding can add tremendous value to a product, but inconsistent and frequent renaming will confuse customers and hurt the recognizability of a product. An example that comes to mind is the product we know today as Skype for Business, this product received a new name with every single release. Another bad example is the recent rebranding of the Exchange web interface to Outlook on the web. I always grin when I find another ‘Outlook Web App’ in an Exchange 2016 TechNet article. It looks like even the technical writers have a hard time keeping up with the name changes. I can’t blame them.

Wednesday, October 5, 2016

Focused Inbox for Outlook is delayed

When Microsoft announced Focused Inbox for Outlook and Outlook on the Web (OWA) in July they planned to release the new features to First Release customers starting early September 2016. Roll-out to the 4th ring of customers was scheduled for October.

image

One month between First Release and GA may seem much, but for organizations that need some more time to understand the impact and communicate the changes with the end-users, a month isn’t that much time.

But more importantly, September has already passed and we have not seen the new feature to appear nor any new updates on the Office Blog or documentation for administrators.

Last week I attended Microsoft Ignite in Atlanta and had the pleasure to speak with some members of the Outlook time. My understanding is that the feature is ready, the documentation has been written but the actual deployment has been rescheduled to the November/December timeframe.

Microsoft has promised to do a better job than they did with the roll-out of Clutter. Documentation for admins should be published before launch to First Release tenants. If you want to be prepared, read up on Focused Inbox admin controls in my previous article.

Thursday, September 22, 2016

Exchange Online and the new email address limit

Exchange Online, just as any other cloud service, is a shared environment where resources are pooled between multiple tenants. This means that certain limits need to be enforced, either to ensure that the services is being used as intended or to prevent that some users consume an uneven share of the available resources.

Luckily most limits, not all, are documented quite well in the Exchange Online Limits section of the Office 365 Service Descriptions. One of the tables on that page contains some limits with regards to recipients. See the following screenshot of this table as it was on the 10th of July, 2016: (click to enlarge)

image

And here is what recently changed:

Capture

A new Recipient proxy address limit was added to the table and immediately enforced.

Interesting is that the column for Exchange 2013 is populated with the value of 200 now too:

image

Unless a recent CU introduced a hardcoded limit I don’t think this is accurate. By my knowledge the real limit in the on-premises world is the character limit of the proxyAddresses AD attribute.

Now this may not apply to you, but there are an awful lot of people out there who have up to 300 or more proxy addresses. Some users created custom addresses for each mailing list of vendor account as Exchange never implemented a wildcard email address feature (jetzemellema+amazon@gmail.com).

And to make matters worse, the admin interfaces do not allow to remove individual email addresses and then save the object again. A possible work around is to export all proxy addresses to CSV, remove them all, clean up the CSV to contain <200 entries and add them again with PowerShell.

The easiest long-term solution appears to be to add additional Distribution Groups where your mailbox is the only member. Now add a bunch of those addresses to the DG to ensure you can still receive all messages sent to the addresses.

In hindsight this would’ve been a perfect topic for Microsoft to announce before implementing the change, including guidance for customers who are impacted by this change.

Make your HCW experience even more fun

Ever wondered what happens when you click through the new and shiny Hybrid Configuration Wizard? Wouldn’t it be awesome to be able to see what happens when you wait in real time? Now you can.

image

People with a background in Unix or Linux are probably familiar with the tail program. tail reads the output of a file and keeps doing so when the file is updated with new data. This is an ideal tool to view log files in real time.

PowerShell offers similar functionality in Get-Content with the -Wait switch. With that in mind, all we need to do is find the most recent log file in the directory as every instance of the HCW creates a new log file and then read the contents of that file.

Start the HCW first, we need the log file to be there before we can read it, and then enter the following one-liner in PowerShell:

Get-Item "$ENV:appdata\Microsoft\Exchange Hybrid Configuration\*.log" | Sort LastWriteTime | Select-Object -Last 1 | Get-Content -Wait

You like that? Then try using Get-Content -Wait against C:\ExchangeSetupLogs\ExchangeSetup.log the next time you’re installing or upgrading Exchange. Have fun!

Sunday, September 11, 2016

How to fix ALT+S in Firefox

Key combination ALT and S is commonly used to save or submit data in web application, such as Exact Online, phpBB and vBulletin. Since Mozilla Firefox 2.0 this is not working anymore.

This can be fixed by editing two settings in the advanced settings of Firefox.

  • In Firefox, visit about:config
  • Change ui.key.chromeAccess to 5
  • Change ui.key.contentAccess to 4

image

The changes are immediately effective, no need to close and reopen the application.