by gustaf | Feb 8, 2018
A quick one today…
Needed to delete a couple of million records for a customer and the natural thing was to use the Bulk Deletion service, well, I turned it on and it was extremely slow. Only got about 10 records/s which would cause the entire delete to take over a week. I have checked with Microsoft and this is not a bug, but it is working as designed and is not designed to be super fast. According to Microsoft bulk deletion jobs are put on the async queue on low priority to allow other more important jobs higher prio.
And a favorite quote of mine from Purvin Patel of Microsoft “Does a dump truck need to outrace a Ferrari?” – and I think that the answer to that question is: it depends. Sometimes it does.
Personally I would sometimes like it to be as fast as possible when removing a lot of records.
I also checked to see how fast the deletion would be with SSIS and Kingswaysoft. Used the following settings:
- VM about 5 ms from the Dynamics 365 instance (important that it not be too far, use an Azure VM for this)
- Used 64 threads
- Used Execute Multiple batching with 10 (cannot use more that 10 if you are using a lot of threads, ie more than 2)
- VM has 8 virtual cores and 32 GB memory
- Loading in batches of 2000. Only loading the id-column, as that is all that is needed.
With this setup, I got somewhere around 345 records deleted per second. Which is a tad more than 34x faster than the bulk delete.
So, want to delete a lot of stuff, maybe Bulk Delete is not the way to go. Not yet anyway, let’s hope Microsoft makes it faster!
(this post was updated on Feb the 9:th 2018)
Gustaf Westerlund
MVP, Founder and Principal Consultant at CRM-konsulterna AB
www.crmkonsulterna.se
by Gustaf Westerlund | Jun 24, 2013
If you have been reading my blogs previously you might have noticed that I several times (More on the insecurity of data, External posting on if your cloud system is safe from the law) have written about the fact that I think that many companies are viewing clouding a bit too lightly, especially from the legal perspective that many countries governments perceive themselves of having the rights to your data if the data either resides in that country, travels through that country or is owned by a company in that country.
From this perspective, I must conclude that the Snowden case is not very surprising. It mearly confirms what I expected to be true all the time and I would say that anybody being surprised is rather showing exceptional naivety towards our governments.
Hence I would just again remind you, that if you have sensitive data, beware of all the threats to the data. There are hackers, lawyers, FBI agents, and XYZ-agents and you have to assess the interest of all potential parties to you data. Then understand and handle the risk in a controlled manner.
Gustaf Westerlund
MVP, CEO and owner at CRM-konsulterna AB
www.crmkonsulterna.se
by Gustaf Westerlund | Jan 29, 2013
As I previously discussed in a guest posting at Software Advice, the legal aspects of cloud computing are interesting and an article recently in “Computer Sweden” (Swedish) again raised this issue with references to the law at hand in the US.
The article references the law Foreign Intelligence Amendments Act, FISAAA, and describes how it can be used without any court order or by-case permits required. According to the interpretation of the law it has some limitations but can rather freely be used to gather non-american data. It does not have to be political data, but can just be data from a forign region affecting US foreign affairs.
As it does not require case-by-case permissions, the interpretation of the law is probably handled quite far down the ranks were this is deemed necessary. The interpretation could hence also be quite wide and I would not be surprised if data such as defence industry business opportunities fall within this area and probably other related areas. Foreign affairs is a wide definition, automobile export, telecomunication equipment and software export is defintley within the boundaries.
I would hence, strongly advice against putting data in countries with legislation similar to this (USA is definetly not the only country). When using the Microsoft Dynamics CRM Online service in Europe, the case is a bit better as the data is stored in countries within the EU:s juristiction. The US law is, however, interesting in this part as it focuses on the companies being american and not where the data is actually stored. Hence, the fact might be that the US government might be able to push Microsoft/SalesForce/Google or any other american cloud systems supplier into handing over data backed up by this law even if the data is stored in other countries.
To be on the safe side, from the legal aspect, storing the data in your own servers run by your own people, is always the safest. Their loyalty lies with your company and you have control over the physical storage of the data. Do not that this, however, is no safeguard agains hackers.
Gustaf Westerlund
MVP, CEO and owner at CRM-konsulterna AB
www.crmkonsulterna.se
by Gustaf Westerlund | Feb 10, 2011
I have been asked by the very well renowned blog at Software Advice, to write a guest post and you can now read it at their site: Software Advice. It is a post concerning the legal problems of cloud based systems and where its data is stored. It is most uncertain which laws apply to the data, and the most probable outcome is that it will be viewed as subject to the country where it is stored. Something that might not always be beneficial to companies and organizations with sensitive data.
Please read it and leave your comment on the subject, it would be very interesting to read your view on the subject. I personally think it is very interesting and that it is one of the aspects of cloud computing that I feel often is neglected by companies.
Gustaf Westerlund
CEO, Chief Architect and co-Founder at CRM-konsulterna AB
www.crmkonsulterna.se
by Gustaf Westerlund | Dec 21, 2010
The Cloud is hot. Everybody seems to be talking cloud computing like it is the holy grail of computing. I try to have a bit more cold headed view of it since I am often faced with the complexities of integrating systems with each other, I often find that it can be very complex to integrate two systems residing in machines next to each other, adding a level of complexity by placing these machines on the internet does make integration more complex and costly.
The cloud does have its advantages though, for smaller companies, like my own, there is no need to own and run large servers. We, at CRM-Konsulterna, do not run any servers at all. The one server that we actually need, our lab environment, is actually hosted aswell, but on a infrastructure level.
I was tipped by Software Advice about an interesting article on Microsofts push on cloud computing for Microsoft Dynamics. You can read it here: http://www.softwareadvice.com/articles/accounting/microsoft-is-all-in-for-the-cloud-but-what-about-dynamics-1121310/. It addresses some quite interesting points from a Dynamics perspective, not only CRM.
I think that you need to understand the background in order to understand why Microsoft are pushing this so hard. The traditional on-premise deployment type of systems has always been Microsofts strongest area and Microsoft has for several reasons, like risk reduction, scalability etc. to have a business that is partner based. It is also heavily focused on adressing the IT part of customers business, which is natural when coming from their background.
The recent years have shown that companies like Google and SalesForce.com deliver very competent cloud based services and this seriously endagers Microsofts core business model since it shortcuts Microsoft offers by adressing the business decions makers directly and circomventing the IT-departments. This is a outspoken stragegy for companies like SalesForce.com.
So, what Microsoft tries to do is to compete on the cloud market and the on-premise market at the same time while still trying to hold on to their partner network and maintain their loyalty. This is of course quite complicated since many Microsoft partners have made a living by installing and selling Microsoft software. There are new models for cloud based service reselling but it does feel like there is going to be a bit of a downside for many partners.
From our perspective, as CRM-consultants, we are happy to offer CRM in any flavor since our main businesses is not selling the licenses but around helping our customers leverage the power of the system by adapting it to their needs. Hence it does not really matter if it on-premise or in the cloud.
However, from a technical perspective, we do recommend either partner hosted or on-premise since that substantially reduces the pains of integrations and adaptions compared to a Microsoft hosted solution. So, our recommendation to our customers is usually to choose partner hosted as that relieves them of the burden of managing the server etc. and at the same time gives us all the advantages of adapting the system to their needs.
The fact that Hunter Richards mentions about the different architectures of the Dynamics ERP products is true but does not really affect Microsoft CRM since it has a good Cloud platform, even though there are some adaptations that only can be done on on-premise or partner hosted systems, it is a very competent and flexible Cloud system and the new version CRM 2011 is even better.
It will also be interesting to see how Microsoft will mange the partner channel in the future. It is something they, with their current business model cannot do without but at the same time something that slows them down a bit since partners naturally are slower to move than inhouse consultants.
Gustaf Westerlund
CEO, Chief Architect and co-Founder at CRM-konsulterna AB
www.crmkonsulterna.se
Recent Comments