Deleting a lot of records fast

Deleting a lot of records fast

A quick one today…

Needed to delete a couple of million records for a customer and the natural thing was to use the Bulk Deletion service, well, I turned it on and it was extremely slow. Only got about 10 records/s which would cause the entire delete to take over a week. I have checked with Microsoft and this is not a bug, but it is working as designed and is not designed to be super fast. According to Microsoft bulk deletion jobs are put on the async queue on low priority to allow other more important jobs higher prio.

And a favorite quote of mine from Purvin Patel of Microsoft Does a dump truck need to outrace a Ferrari?” – and I think that the answer to that question is: it depends. Sometimes it does.  

Personally I would sometimes like it to be as fast as possible when removing a lot of records.

I also checked to see how fast the deletion would be with SSIS and Kingswaysoft. Used the following settings:

  • VM about 5 ms from the Dynamics 365 instance (important that it not be too far, use an Azure VM for this)
  • Used 64 threads
  • Used Execute Multiple batching with 10 (cannot use more that 10 if you are using a lot of threads, ie more than 2)
  • VM has 8 virtual cores and 32 GB memory
  • Loading in batches of 2000. Only loading the id-column, as that is all that is needed.

With this setup, I got somewhere around 345 records deleted per second. Which is a tad more than 34x faster than the bulk delete.

So, want to delete a lot of stuff, maybe Bulk Delete is not the way to go. Not yet anyway, let’s hope Microsoft makes it faster!

(this post was updated on Feb the 9:th 2018)

Gustaf Westerlund
MVP, Founder and Principal Consultant at CRM-konsulterna AB
www.crmkonsulterna.se

Supported browsers – update

Supported browsers – update

One of my most popular posts is a collation on which browsers are actually supported for Dynamics 365 CE. This is not strange since it is a common issue, we often have customers call our support complaining that Dynamics doesn’t work well, and as described in the original article, the cause is usually that they are using an unsupported browser.

Also the Micrsoft Technet article for this: https://docs.microsoft.com/en-us/dynamics365/customer-engagement/admin/supported-web-browsers-and-mobile-devices
and the detailed version: https://technet.microsoft.com/en-us/library/hh699710.aspx mainly focuses on Edge and Internet Explorer compatibility for the different windows versions. And despite Microsoft’s hopes, the rest of the world have not totally turned to all of their programs, yet.

There are currently some issues with version 9.0 as well, why I thought it might be a good idea to write a new article on which browsers are the supported one in a easily overviewable grid.

A brief explanation to the different abbreviations used in the picture:

  • SW – Supported and Working.
  • SW* – Supported and Working on OS X as designed but with somewhat limited functionality. If you compare Dynamics on OS X running Safari with it running Edge on a Win 10 you will notice that there are more options.
  • SW** – The app is not the same thing as the browser at the moment. The Unified Interface will make them almost the same. There will still be differences, in that the app will have better integrations to the device’s hardware, like camera, etc. Especially for the earlier versions, the app was very limited. It is more powerful know. For field service there is also a special Field Service App which is not as HTML/JS based (it is based on Resco).
  • SW*** – I have not had the chance to test if version 9.0 works on a Nexus 10 running Chrome but the official documentation says it should work. However, as v.9.0 does not work with Chrome at the moment, (this will probably be fixed very soon) I would actually be surprised if it worked for Chrome on Nexus 10.
  • NS – Not supported. This does not mean, however, that you cannot get it to work. I have heard of people that mange to get Chrome on OS X to go into some “PC” mode which tricks Dynamics. This still doesn’t make it supported, but it might work for you. The problem might be if you have any issues, that Microsoft Support could leave you hanging.
  • SNW – Supported but Not Working – As for Chrome on Windows, the general story is that Microsoft only support the latest version of it. However, despite version 9.0 official supporting Chrome on Windows, it doesn’t seem to actually work. I would presume that they will fix this soon. The problem is that Dynamics 365 won’t actually load.
The support matrix for the browsers is generally a tricky question. The matrix above is also simplified a bit as I have not included the parameter of the different versions of Operating systems. It can probably be presumed that Microsoft will make greater efforts into making sure that the latest and possibly even some versions before that, that they know customers are using for Edge and Internet Explorer, work well with Dynamics 365. I know some of you will probably think I am crazy for even mentioning Internet Explorer, but I have several customers who still have IE as the company standard browser, and it is not due to Dynamics 365.
As for Firefox, Chrome and Safari, these browsers might be very popular, but from a Microsoft perspective I can see a very tricky situation, and that is that they roll out changes without notifying anyone. This was, for instance, done by Google a couple of years ago when they changed their handling of modal dialogs. This caused many dialog handling features of Dynamics CRM (at that time) to break. It took some time for Microsoft to identify the change, fix it, make sure there were no regression errors, roll it through all testing scenarios and deployment stages and controlls they do until they were able to get a fix out. It did come out quite fast, but there were some people nagging at Microsoft for having a bad product, which I personally find a bit unfair.
The essence of the two paragraphs above is; feel free to use Chrome, Safari or Firefox but I would suggest always having Edge and/or IE as a backup option in the case that an update comes to the main browser that severely affects the usage of Dynamics 365.
And I hope that Microsoft soon fixes the error that stops us from using Chrome with ver. 9.0. Any day now!

Gustaf Westerlund
MVP, Founder and Principal Consultant at CRM-konsulterna AB
www.crmkonsulterna.se

Dynamics 365 Support – your best friends when you have Dynamics 365 Online

Dynamics 365 Support – your best friends when you have Dynamics 365 Online

If you are using Dynamics 365 Online, Dynamics 365 Support is your IT-department for your Dynamics system and you need to know or start learning how to working with them in a good way. There are several things you should consider.

Registering a support ticket

First of all, registering a support ticket is of course important. This might not be as trivial as it seems, as it depends on which type of license you have. If you have bought your licenses using CSP (Cloud Service Provider) or Select/Volume Licensing then you will most probably be directed to your CSP/Select partner for any support issues. You can of course buy premier support from Microsoft but that might not always be an option for smaller companies. There are some tricks as well if you are stuck with CSP/Select or similar, and will be happy to tell you about these in person if I meet you, but I won’t write about them, as I am afraid it will be “fixed” which would be sad.

If you have bought your licenses directly through the O365 portal, you can also create a ticket using the portal. You need to be a Global Admin or a Dynamics 365 Service Admin to place a ticket. Currently you cannot indicate that the ticket is for Dynamics so you simply write so in the text and the O365 support team will create a new Dynamics 365 ticket for you. I was amazed at how fast the Office 365 support called me after registering a ticket, I think it took less than 1 minute. So I would suggest you choose the option “call me”.

If you have a partner, I strongly suggest that you register them in the Office 365 portal for the Dynamics 365 subscription as Partner of Record (PoR). This will make your company show up in their Microsoft partner portal where they can register a support ticket for you.

Checklist
When registering the ticket with Microsoft, there are some things you have to note:

  1. Be polite. Even if your system is slower than sirup in January (a Swedish saying) or nobody can log in, you will not be getting any help faster by screaming or being rude. I like to quote my grandfather who had a good saying: “Better to be strong in argument and withheld in words, than the other way round.” I have actually gotten written compliments for being nice by the support people, do you think they will go the extra mile for me?
  2. Be very exact with what is you think is wrong. The best way is to formulate it like this: “I did X and expected Y but I got Z”. Also be clear on when you define the case as closed. This is particularly important, and hard when you have performance issues or intermittent errors. But try to be as exact as you can. For example: “When users run Chrome open a Contact (after the contact form already has been loaded for another record) it takes 10 seconds. We expect it to take no more than 2 seconds.”
  3. Only have one issue per ticket. Better to have many tickets open. I have had 4 tickets open for one single customer at the same point in time.
  4. If you think the issue is important, describe it in business impact terms. For example “The bug X affects all sales people in the organization effectivly stopping them from using Dynamics 365. It severly risks our entire Dynamics 365 investment as users look at other solutions and the user adoption drops dramatically and the trust in the system is drastically lowered for every hour this bug is not fixed. We have 4354 sales people in our global sales organization spread out over 34 countries. It also affects managements abilities to manage the company as no pipleline and forecast numbers are produced which could potentially cause sever business damages that cannot easily be fixed within the coming 4-5 years.” – This was just made up 🙂 but it is important to not hold back when doing this, especially if you are Swedish or from other cultures which value humbleness.
  5. Be clear on limits of the error. 
    1. Which Operating system?
    2. Which browsers?
    3. What times? Can you see any clear patterns on when it happens?
    4. Geos?
    5. Security roles?
    6. Types of users affected, AD users federated to O365 or Cloud users?
    7. How long?
    8. Which versions of Dynamics 365 are you running?
    9. Which instances have the issue (the url:s)?
Types of tickets
What types of tickets can be registered? Should you only register a ticket when you want help with something?
Well, no, you can and should register a ticket for many different reasons. As I mentioned at the top of the article it is important to think about Microsoft Support as your Dynamics 365 IT-department. They will help you with a lot of things that your IT-department would have helped you with had they hosted your Dynamics system. The main difference is probably that they do nothing else than host Dynamics 365, every day, year after year… not many IT-departments that can compare with that.
Typical issues they can help out with are:
  • Something seems to have stopped working that used to work, and you didn’t change anything. Like you cannot create an Application User anymore.
  • Something clearly is a bug in Dynamics 365 and you want Microsoft to know about it (they might not know) and you want to let them know how important it is for you (they will surely not know this).
  • You need some information from the Server, like the “Top 10 table Usage”  report from the SQL database.
  • You have performance issues.
  • You need to change the planned upgrades because your team won’t make the set time.
  • You want to set some non-clustered indexes on the database.
  • You are doing a migration of large amounts of data and you will be a bad neighbour for a few days and if possible you would like a little more than the 2 allocated threads in the API.
  • You want a database copy prepared for installation into an on-prem equivalent.
  • You need some detailed information on how the service is managed, like
    • How backups are stored
    • How to restore backups from the 90 day tape backup
    • Where an instance is hosted
  • You would like to move an instance to another datacenter
  • You need to remove a managed solution but it has dependencies that shouldn’t be there…
  • You need to do some task with an instance that you cannot do yourself with the instance manager.
In other words, they will be your superfriend when working with Dynamics 365 and you want to be really nice to your super friend!

Gustaf Westerlund
MVP, Founder and Principal Consultant at CRM-konsulterna AB
www.crmkonsulterna.se

Working with CrmSettings via PS – setting things you didn’t know you could, or should

Working with CrmSettings via PS – setting things you didn’t know you could, or should

Recently I have been trying to fix some Server Side Sync issues with an on-prem Dyn CRM 2016 and this got me to dig into the nitty gritty of settings for the system.

Before going berserk and chaning a lot of these settings, be aware that you can cause severe problems with your system if you are not careful so I would strongly recommend that you test any settings and also export any settings so that you know what they are.

Finetuning your onprem installation is something that you can do with an onprem that is a lot harder with online.

By logging in on your CRM-server, and opening Powershell and entering the following command, you can start working with some settings that are otherwise hard or impossible to reach:

Add-PSSnapin Microsoft.Crm.PowerShell

For instance, if you want to check out some of the ServerSideSync Settings, enter:

Get-CrmSetting -SettingType ServerSideSyncQueueSettings

If you are using the Windows Power Shell ISE, after loading the Microsoft.Crm.PowerShell, you will get intellisense that will help you see which different settings are available. So if you write:

Get-CrmSetting -SettingType

And then enter a space after this, you will see the different options:

For some more details into Server side sync and how to configure this in detail, look here:
https://download.microsoft.com/download/7/1/D/71D44C6E-CB78-4573-BDFF-77A6E137BA5C/serversidesyncwithDynamicsCRM.pdf

And here is an example of how to set data.

$set = Get-CrmSetting -SettingType ServerSideSyncQueueSettings 

$set.MailboxQueueItemsInMemoryHigh = “500” 
$set.MailboxQueueItemsInMemoryLow = “200” 

Set-CrmSetting -Setting $set

Just add more $set – statements to set more values.

If you want to write some settings to a file, just pipe the output to a file, for example:

Get-CrmSetting -SettingType ServerSideSyncQueueSettings > ServerSideSyncSettingsOrg.txt

But as I mentioned above, be careful with these settings.

Gustaf Westerlund
MVP, Founder and Principal Consultant at CRM-konsulterna AB
www.crmkonsulterna.se

First full day at CRM UG Summit 2017!

GDPR and October Update of Dynamics 365, that is what has been the main take away from today for me at CRM UG Summit, (UG is User Group for those who do not know.)

I have discussed GDPR a lot with a lot of different people here, some more in tune with what it is about than others and some not even realizing that it will affect all companies in the US having EU Citizens in their systems.

And of course the new release of Dynamice 365, called October Update or version 9.0 for those of us who like to be clear about what we are talking about.

I will come back to some of the details, for now I will leave you with some recordings I made.

Gustaf Westerlund
MVP, Founder and Principal Consultant at CRM-konsulterna AB
www.crmkonsulterna.se