Unable to upgrade CRM 4.0 database more than once

My colleague Rickard had an interesting problem yesterday when working with a client of ours. They had a prototype upgrade and now wanted the production upgrade. However, they wanted both of these organizations to be available (yes, will required an Enterprise license or separate SPLA licensing if it is a hosting environment). The prototype upgrade they want to be available as “orgname-old” and the normal upgrade just as “orgname”.

An interesting issue arose when Rickard tried to import the CRM 4.0 database into the CRM 2011 environment he receive an error message saying that a CRM 4.0 database can only be imported once into a CRM 2011 deployment. This might be noted somewhere but it is not something that I or my colleague Rickard have ever seen when working with CRM and we didn’t find to much about it googling the issue either.

The problem seems to be that Microsoft seems to have cut a corner when upgrading a CRM 4.0 database in comparison to normal redeployment as it does not give the organization a new GUID. Hence you cannot import two organizations twice.

The workaround is, however, quite simple, upgrade the first database, disable it and remove it from deployment manager (do not delete it from the SQL-database).

Upgrade the second database into CRM 2011. As there is no organization with this GUID now, this is no problem.

Now, import the first database (redeployment) that you previously disabled and removed from deployment manager. You can now give it a new name if you like. As this is a redeployment, it will give this a organization a new Guid and the two organizations will not conflict.

If you are creative you can probably think of you own solution using similar techniques.

Big thanks to Rickard for input to this posting!

Gustaf Westerlund
CEO, Chief Architect and co-Founder at CRM-konsulterna AB
www.crmkonsulterna.se

Problems with converting email to case

Problems with converting email to case

I was recently working with a customer when I noticed an interesting bug in CRM 2011, in the Outlook Client. When trying to convert an email to either a lead, case or opportunity directly from the Outlook ribbon, it does not set the “regarding” lookup field of the email that was tracked. However, if the email is first tracked, then “viewed in CRM” and then from the CRM email form converted to lead, case or opportunity, the regarding field is set correctly.

To make this a bit more clear I will describe this with a few screenshot below:

We’ll start off with opening a received email and pressing the Track button in CRM

This will then allow us to press the convert to Opportunity, Case or Lead, directly from Outlook. A very neat, and new function. It came with CRM 2011. In this example, I am selecting Case.

 This will show a smaller dialog allowing you to select case (not shown here) and also prompting you if you want to open the case or not. I selected to open the case.

After opening the case, I checked the closed activities to see if the attached email that was I just converted to a case, but as you can see below there were none. I was a bit perplexed as I was showing this new functionality to the customer as was expecting to see the email here.

I went back to the email from Outlook and opened it in CRM and found that it had not been correctly connected to the case using the lookup “Regarding” field.

So, I tried it the old way. I sent myself another email, pressed the track in CRM button, and then instead of converting it directly from Outlook, i pressed the “View in CRM” button to open it in CRM.

I then converted it to a case from the ribbon menu from the CRM Email from by pressing the “To Case” button.

I was shown a similar small dialog letting me select subject for the case and similar and then the case form was shown. It was identical to when created from Outlook, but when checking the closed activities there was one important difference;

And by opening the CRM email form from the list shows it clearly:

The regarding field has been set properly.

This is a very unfortunate bug in CRM and I do hope Microsoft solve it quickly as the feature that CRM 2011 adds in Outlook by being able to convert emails to cases, opportunities and leads directly from the email form is very good but as all of you know who work close to sales or customer service people, their time is very precious and every click counts and new features that do not deliver as expected are always annoying to the user. So, until the bug has been fixed, make sure the users open the CRM form to convert the emails.

Update: Jukka Niiranen left a very informative comment to this posting below informing us that Microsoft probably will fix this bug in UR6 planned to be released in Jan 2012.
Gustaf Westerlund
CEO, Chief Architect and co-Founder at CRM-konsulterna AB
www.crmkonsulterna.se

Built in imports in CRM

I have been working with the built in import functionality for imports a bit lately and I must say that it has been improved a quite a lot. It as now reached a level where it is quite usable but there are still problematic areas you have to look out for.

I found this blog posting (http://blogs.msdn.com/b/crm/archive/2010/03/15/importing-multiple-files-using-zip-import.aspx) which was quite interesting and enlighted me to the fact that you can import zip:ed files which is really good since there are some size limitation on file uploads to CRM. Since you usually get really good compression rates on csv and xml files this is really good news.

For those of you who don’t know these limitations, the maximum upload file size to CRM is 8 MB. But if you zip the files the total size of files that can be contained within the zipfile can be up to 35 MB. Just to give you an example of the compression rates that can be achieved, we tried compressing a 70 MB xml based excel document and the zip-file we got was only 0.5 MB. That is a ratio of 1/140, less that 1% of the original size.

Zipping the import files is also good for uploading multiple files with at once, for instance when the files have dependencies or when there are attachments as this reduces the manual work a lot.

A bit of advice though. I would strongly advice against using the csv-format since it is very volatile and often brakes and gives rise to a lot of strange errors. It is also dependent on the local regional settings, for instance, in Sweden, it is not comma separated, but semi-colon separated, as this is the standard numeral separator. Very confusing.

On top of this you often have problems with codepages, if you work in other languages than english, and you have to re-save the file in UTF-8 to make it import properly. Today we also had some issues with Excel insisting on formating the Swedish personal number (Social Security Numbers) as 7.41231E+12 instead of the correct 7412311234. We had to manually force it with different kinds of formatting to save the number correctly.

It didn’t end there, Excel also found it fitting to add a few empty columns to the end of our CSV-file making it impossible to import into CRM.

So, we have been working to iron these things out and we were getting there but somewhere along the line we decided to skip the CSV and to go all in for the excel as xml format instead. The files that are saved are a lot bigger but with the zip feature it is managable.

Despite the fact that the new xml-format support and the zip-support exist I think the next time I will be even more vigourous in my recommendation for an import tool such as QuickBix Integration Suite, Scribe, or Import Manager. The new import features are great but for smaller imports, not for migrations.

Gustaf Westerlund
CEO, Chief Architect and co-Founder at CRM-konsulterna AB
www.crmkonsulterna.se

Nice blog posting on new CRM 2011 certifications

Back from the summer vacation and discussed CRM 2011 certifications on www.crmforum.se where my friend Fredrik Neiderud hinted about a nice blog posting by Richard Knudson regarding the new CRM 2011 certifications and what new in them.

Please read about it here: http://www.dynamicscrmtrickbag.com/2011/07/11/dynamics-crm-2011-certifications/

Gustaf Westerlund
CEO, Chief Architect and co-Founder at CRM-konsulterna AB
www.crmkonsulterna.se

Some interesting questions on xRM functionality

I have received a few questions from one of the readers of this blog and I will answer them bellow:

I am currently working on a project which is planning to use CRM 2011 as a development platform for migrating an existing  vertical specific LOB application.  As the existing application has a large and complicated database, I am trying to better understand how CRM 2011 can use external data, hence the following questions:
          Can CRM 2011 use external data assuming it is exposed as an OData feed?

Yes and no. CRM 2011 cannot using some simple mode of configuration connect to an OData feed but there are numerous customizations that can be done using techniques such as javascript, mash-ups, SilverLight, XSLT or ASP.NET that can be used to work with the Odata feed. When I work with my clients, I usually try to understand what they are trying to achieve to be able to recommend a technique that fits their requirements.

          Will CRM allow creation of Metadata for external Odata?

Yes, as mentioned above the different customization points allow for different ways of communicating and working with external data and metadata such as OData. There are no built in methods that automatically will connect OData to CRM (as I know of anyway).

          Can CRM security model be used for external Odata?

The CRM security model is rather complex as it incorporates several concepts at the same time;

o   Security roles
o   Organizational context with ownership of objects based on this and in conjunction with the roles.
o   Sharing privileges to individual users and/or teams

This results in the fact that it is rather complex to replicate the privileges for a certain object to a related object in another system. It can be done, but requires in-depth knowledge of the CRM security model and the API:s as all data related to this can be accessed from the API.

As an example of the complexity, let’s say that you have users A, B and C. User A belongs to division inner sales, user B to division outer sales and user C to division service. All have the same role that only allows access to the accounts within the same division. The account “Vodaphone” is owned by A and A creates a new team to which he shares the rights to view the account “Vodaphone”. He then adds B to this team while creating it. Some time later he adds C to this team as well and shares read-write access directly to B as well.

This is an example that despite the fact that they all have the same roles would involve a lot of setting access rights in the external system since the adding of team members to teams indirectly sets their rights. It is not impossible to implement but quite complex and does require a considerable amount of effort. For an experienced CRM developer, probably not less than 200 hours.

When discussing issues like this with my clients, I, as I usually do, try to understand their core needs and motives so that I can suggest a good-enough solution that will give them the best bang of the buck.

          Does CRM allow the use of external Odata in native forms or custom Silverlight UI has to be developed?

No. Fields (previously called attributes) are directly connected to the database and there is no support for virtual attributes that will retrieve data from an external source.

Integrations to external systems, using any data source, for instance OData, can of course be created that will transfer data to CRM. Data could for instance be periodically sent or sent when there is an update if the external system allows that kind of triggering.

It is also possible to jack into the retrieve or retrieve multiple events that occur when CRM tries to retrieve data so that CRM could pull data from the external data source when data is retrieved. I would not recommend this though since these events occur a lot and it would probably affect performance badly. It is also supported to fetch data directly from the database views using SQL and that would bypass this logic making this data incorrect. This would also cause some problems with working with CRM offline.

The solution you are hinting at using Silverlight is of course also an option but would be similar to some mash-up integration that could also be created using ASP.NET or javascript.

It would sometimes also be an option to create specific reports in CRM that connect to external data sources and integrate the CRM data with the external data in this manner. This method is preferable when using data warehousing or when there are heavy performance issues, as these reports can be cached and scheduled.

Without knowing more about the specific requirements, I would hence recommend some kind of periodic or live integration that transfers the required data to CRM in read-only fields as this would be most in line with how CRM works and would enable all built in features that CRM has to work with this data. It would also be best from a performance perspective and could be scaled if needed.

          Can external Odata be used in Workflow activities?

Yes, custom workflows activities can be created in all versions of Microsoft Dynamics CRM 2011 except CRM Online where the same logic can be created with some work-around effort using plugins instead. These custom workflow activities are created using the .NET framework typically in C# and can hence easily use any available datasource, like Odata.

On a related front, does CRM allow complicated (i.e. highly normalized and multi-level) data hierarchies to be implemented  natively?  If so, is there a performance penalty?  Would we be better off using this data as an external OData feed or bringing it internal to CRM?

No, CRM does not support some of the more complex OO concepts like interfaces and inheritance. As this is something many of us in the CRM community are looking for and it would also be a great addition to the xRM concept so I am certain it is on Microsoft’s list of features they are trying to implement. How far up, I cannot say.

This is usually not a major issue for me when I work with my clients and I have created very complex verticalizations, for instance for the Telco industry which implement a highly normalized and multi-level information model called SID, in CRM. What you need to do is make it a bit more flat and more concrete which you more or less have to do anyway if you want to make the system usable. It is of course important to be well aware of the simplifications you are making but I do not imagine that any usable system possibly could implement the entire SID information model without simplification and still make the system usable and with any reasonable budget. The abstract class “Party” for instance is just too abstract to be of any real use, but it is of course useful to know that contact, account, competitor, lead, system user etc. do share some information. The implementation would however have benefited from one or two inheritances or interfaces.

If the data should be kept in or outside the system is not easy to answer without deeper knowledge of what you are trying to implement. If you keep it mostly outside, you would have to define a very strict line at which information is kept inside or outside of CRM. Do not, however, underestimate the power of the xRM platform, it is often advantageous to implement large or all parts of the LOB application inside CRM/xRM since you get so much functionality for free when doing this.

In general I would need deeper domain knowledge to be able to give you more direct recommendations and I do not think that this is the forum for that kind of discussion as that requires a dialog and would probably best be done in workshops.

I do like the fact that you sent me these questions as I think they are very relevant when looking at the system from an xRM perspective and put the finger on some of the issues that Microsoft have to improve in the system. If there are other readers out there, if you have questions, please send them to me and I will try to answer them on this blog.

Gustaf Westerlund
CEO, Chief Architect and co-Founder at CRM-konsulterna AB
www.crmkonsulterna.se