New API Limit

New API Limit

by | Feb 25, 2018

Photo by Vidar Nordli-Mathisen on Unsplash

Related to my last post, on working with the API quickly, Microsoft have now released official
documentation that they will, effectivly March the 19:th start limiting the number of API-calls per instance that is allowed to stop what is called “noisy neighbour” problems.

First of all, read the full article here: https://docs.microsoft.com/en-us/dynamics365/customer-engagement/developer/api-limits 

Let’s break this down a bit, 60 000 calls per 5 minutes, translates to about 200 calls per second. If you break this, you will start getting exceptions, until the 5 minute period has ended. You are expected to back off, and essentialy handle this. That is the short version. Read the full article for more details.

Update: George Doubinski, a friend of mine and one of the brains of CRM Tip of the Day made me aware of the fact that the limit is per user. I will update the article below on what this means.

What does this mean? Is this a problem?

For most organizations, no, at least that I work with, I not even close to breaking this. If they are using some integration tools like Kingswaysoft or other tools which enable multithreaded integrations, but generally do not need that kind of data throughput then you might temporarily be shut down, but it should self heal after some time, as after each 5 minute time span, you will get another 60 000 requests. That could probably quite easily be fixed by checking the settings of the integration tool. Update: Also, if you integrate each system using a separate accont, you do not risk one system temporarily blocking many other systems from integrating to Dynamics 365. If you are using normal users, this will of course entail a certain license cost, why I generally recommend using app users for integrations, if possible. And after this, you should have one app user for each integrating system.

However, there are some organizations where I forsee issues, and these are organizations which have combinations of any of the following criteria:

  1. Third party products which, like Marketing Automation, (ClickDimensions, FreshRelevance, SalesForce Marketing Cloud) which have not had time, or got this in their scope yet, and have large amounts of data that they integrate into Dynamics 365. Update: Especially if the user they are using to integrate, the service user, is a normal user, either used by a normal user, or shared with integrations with other systems.
  2. Legacy Code that has been upgrade to new SDK but uses inefficient architecture – can for example have issues with using ExecuteMultiple which in the article above is described as the recommended best practice. Typically for the reason that the architecure of the code, would require major rewriting to allow for ExecuteMultiple. Update: In this case I strongly recommend looking at using a dedicated user for this specific integration, to isolate any limiations set on that user.
  3. Organizations with multiple heavy integrations to Dynamics 365. Will be hard to control that the sum total does not exceed 60k per second, and handle back-off in a controlled way. The only reasonable way would probably be to rewrite the integrations to use a proxy or queue instead like Azure Service Bus Queues to integrate and have a single integration interface. Probably a lot easier to write in a blog article than to do in real life. Update: This was an incorrection deduction from my part, as it is not based on the sum total, but on the sum per user, this is not a risk unless many integrations use the same user for integration which I do not recommend.
  4. Organizations with complex heavy integrations with thousands of lines of integration code that need to be redesigned, rewritten, tested and deployed before March 19:th. And there is no way to test it as there is no TAP/Beta program for this “Feature”. Update: This is still very relevant. Even such a small change as changing the integrating user for an external system should be thoroughly tested and for larger implementations that can be hard to do before March 19.

Example

I see is a typical B2C organization running Dynamics 365 with a marketing automation addon with email tracking and webtracking. They also have a very time critical integration of orders to be able to handle any incidents. Even if the order integration in itself does not reach the limits, it is not unforseeable that a mailblast, especially a good mail blast, to which many customers read the emails click the links, go their site, check their offers and start ordering, would cause a surge of traffic on the Marketing Automation integration – Dynamics 365 API. This of course depends on the settings of this, but perhaps it is critical that all events be tracked to Dynamics. With a mailblast to let’s say 1 Million recipients, quickly hitting the 60 k/5 min limit would happen. When this happens, this would also block all orders from going to Dynamics, causing an effective stop for working with any new incidents in the system.
Update: This is, of course, only relevant if both systems are integrating using the same user. Don’t. However, the marketing automation system above, would hit the limit fast anyway and if the supplier of this system didn’t have time to update their product/service then it would handle this incorrectly. I recommend checking integrating systems and try to turn down the verbosity of what they are writing to Dynamics 365. Then after March 18 when we see how this falls out in detail, you can test a more verbose setting in a test environment, and then see how that falls out.

Summary

For small and medium companies with low complexity working mainly with B2B. I don’t see that much of a problem. Larger companies with complex integrations, large databases, integrations to webtracking, email tracking which often will be B2C companies which have higher levels of automation and larger databases of customers, will probably have larger problems with this and need to start think about this right now.

We need to come back to this subject post March 19, to see how this will really work. But I think the real problem will be for the larger orgs with many and heavy integrations.

I would be really glad to hear your views on this like I got Georges’.

Gustaf Westerlund
MVP, Founder and Principal Consultant at CRM-konsulterna AB
www.crmkonsulterna.se

Dynamics 365 Version 9.0 Business Edition and more

Dynamics 365 Version 9.0 Business Edition and more

by | Jun 28, 2017

Version 9.0 is soon here and it is set to be the largest release ever from Microsoft. The biggest news on my horizon is the relase of the all new Business Edition of Dynamics 365 (BE). This will be an interesting new product and I’d thought I’d write a few lines about it just to clear some things up based on my understanding of the product.

Note that these are my thoughts, they are based on what I have heard and what information I believe is in the public domain (non-NDA). If you know that anything is wrong or have something to add, please leave a comment and I will try to fix it as soon as possible. Due to the high level of spamming I have to use moderation on all comments.

Is it xRM?
Yes, it is going to be based on the xRM platform but it will be using the all new and shiny “Unified Experience” UI which is probably going to be great in many aspects, looks great and everything but also will probably have a glitch or two as it is just coming out the factory doors.

Ok, so can I customize it?
Yes, to some extent you will be able to. It will have specific limits that set it distinctly appart from Enterprise Edition (EE). As of now Microsoft have not been very specific on the details but they have said that the limits will be on:

  • The amount of users in the system
  • Number of custom entities
  • Max amount of GB of usage on disk
I am an ISV developer. Will my ISV 3:rd party addon work with BE?
As far as I understood Microsoft, they will limit ISV addons to only be able to work with the OOB entities in BE. As creating entities in the “normal” CRM/EE has almost no cost, almost all ISV solutions I have seen have at least one custom entity, if not to contain customizations. So, my answer would not be a blanket “no” but a probable, “no”. If your addon isn’t too advanced, you could probably rewrite it to make it work, but with larger addons like for instance ClickDimensions or Resco, I don’t see how that would be possible.

Can I use a sandbox with BE?
No. No non-production instance will be included in the price and as far as I understood, you won’t even be able to buy a non-production or even production instance of BE.
Can I upgrade from BE to EE?
Yes. As it is based on the same xRM framework this will totally be doable on a theoretical plane. However, I don’t know how this will work on a short term basis if, for instance, you start with a BE in Nov 2017 and in Dec 2017 decide that it was too limiting too you and you want to upgrade to EE. The reason for my hesitation in this is that BE will be based on Unified Experience and EE will not at the time of release of 9.0. I would, however, expect 9.0 to catch up further on the line and after it has been upgraded to Unified Experience, then an upgrade of an instance from BE to EE should be no problem.
I bought the interim EE offer for BE pricing level, will I be able to downgrade to BE?
This is a tricky one. First of all Microsoft has not made any public statement on this that I am aware of in either one or the other direction. However, I do think that the key here lies in the level of customizations that you have done to you system. If your EE system falls with the limits of # of users, # of entities and # of GB that is set by MSFT, and also not having any ISV addons not allowed according to BE and any other limitations that Microsoft might come to set before the General Availability (GA) of BE then I think you have a very good chance of being able to downgrade. However, just as I mentioned above in the ISV section, the limitations might be too limiting and creating a lot of entities and adding ISV addons both free and purchased might make it hard to move to BE.
Can I use the Marketing App of BE in EE?
This is a really interesting case, as this is an area where the BE will have a featureset that exceeds EE, at least at the time when it is delivered. In the big overview of all Dynamics 365 Apps (for example the one above) – the Marketing Automation slot is filled by Adobe Marketing. However, for the BE there is a Marketing App which has been created from the bottom up by Microsoft on the xRM platform. Do note that for those of you who were unfortunate to have chosen Microsoft Dynamics Marketing, that product has been discontinued, so that is as of the General Availability of Dynamics 365 (dec 2016) you should really start thinking of moving away from it.
However, the new and shiny Marketing App for BE is strickly speaking only for BE for the time being. I think that the feature set it has is perceived to be too small for the Enterprise segment, although I can perceive many customers using the Enterprise edition that would probably be more than happy with it.
As for the future for the Marketing App in EE only Microsoft knows what they have planned and if I knew I couldn’t have written anything here either. What we know is that they have invested quite a lot of money in building some Marketing Automation features for the BE which is only aimed at the lower end of the customer segment. I would find it noteworthy if they did not leverage that investment in some way in the Enterprise segment as well. However, building a Marketing Automation solution that can match other Enterprise Marketing Automation systems is not something that even Microsoft can do at the flick of the hand, so I would not expect this to show up for at least a year or two. And it might of course be that they will slowly evolve the BE Marketing App until it is fit to grow into the larger Enterprise segment by just turning some switches, but I guess only the future will show.
As of now, Adobe is the preferred partner for Marketing, something I personally would like to see more detail to. What does it acutally mean? Seems there is some integration engine but no availble whitepapers to dowload that describes what it does. http://www.adobe.com/enterprise/microsoft-partnership/digital-foundation-azure.html 
If Microsoft are investing in building their own EE Marketing Automation App, which does not seem very far fetched althought it might take some time, you might need to take this into consideration if you are considering investing in a new Marketing Automation system for your organization and you are already running Dynamics 365.
If you have any other questions on Dynamice 365 Business Edition, please ask them in the comment. Not sure I can answer, but I can try!
By the way, my blog was awarded the 78:th best CRM blog of all in the world in all CRM categories. Hence the new shiny logo to the right. That was worth some celebration!

Gustaf Westerlund
MVP, Founder and CTO at CRM-konsulterna AB
www.crmkonsulterna.se