Toth’s Tales from the Trenches

After wrapping up a project that used content types extensively, I decided to write a series of posts detailing the headaches encountered so far.

This is part 2.

List Column Settings and STP Template Files

When you create a site content type, add that content type to a list, and then wrap that site up as an STP site template, none of your list content type column settings are carried over to sites based on the template. This means that if you've overridden the Hidden, Required, or Optional settings on the columns in your list content type from the base settings in the site content type, when a new site is provisioned from the site template, all the column overrides will be lost.

It appears that when the STP is packaged up, it only contains pointers to the parent content types, and ignores any list-level overrides to the content types.

Figure 1: List Content Type before STP (Notice the TestNonHiddenColumn is overridden here to be Hidden)

Figure 2: List Content Type after provisioning with STP (Notice the TestNonHiddenColumn is set back to Optional, like it is in the base content type)

Note that this is the same behavior experienced with creating List and Document Library templates as well, which must use a similar approach to templating that STP files use for site templates.

After wrapping up a project that used content types extensively, I decided to write a series of posts detailing the headaches encountered so far.

This is part 1.

Hidden Fields and Inherited Content Types

If you create a site content type through the UI and set a column to hidden, then create another site content type that inherits from the first one, and finally add that content type to a list, the column that was hidden will not be available anywhere in the UI on the list. Note that if you just use the base content type directly in the list, the hidden column WILL be visible in the list UI, and can be used in Views and toggled back to Optional or Required. 

Figure 1: Base Content Type

Figure 2: Child Content Type (notice hidden column is visible)

Figure 3: Child Content Type in List (notice missing column)

It appears that Microsoft is assuming that a hidden column in a parent content type should not be able to be overridden once used in a grandchild content type. Whereas, some people might want to use the Hidden option to simply set a default for child content types, which may need to be overridden in particular lists or grandchildren.

The major pain with this is that you cannot regain the visibility of the column through the UI, even if you go back to the parent content type and set the column to Optional or Required. At that point, the column on the list is set to hidden, and has the CanToggleHidden attribute set to false. You have to either delete the content type from the list and re-add it (which will not work if you have list items already using that content type), or you have to resort to reflection to programmatically fix the column.

Ideally, content types created through the UI should have all of the granularity that is present in XML content type definitions. You should be able to specify ShowInEditForm, ShowInViewForm, CanToggleHidden, etc.

I ran into a situation with some custom InfoPath Document Information Panels where I was getting schema validation errors like this one because of a Business Data Column I added to a document library:

"…{guid}ColumnName is unexpected according to content model of parent element 'documentManagement'."

You might run into this in the following situation:

1. You create the DIP on a Site Content Type.
2. You configure a document library to use your Site Content Type.
3. You add a custom column to the Document Library (and subsequently to the List Content Type (the local instance of your site content type that is applied to the list)).

In this situation, your Document Information Panel was created without the knowledge of this extra column, and does not have a field in its data source schema for this column. What I believe is happening is that when attempting to view the DIP, SharePoint is passing all the columns on the document library to the InfoPath form, and the form is not expecting this extra column, so it chokes (very lame, Microsoft).

To get around this:

1. Edit the DIP on the List Content Type (List Settings, click content type name, Document Information Panel Settings, Edit this Template link)
2. Once the form loads in InfoPath, choose Tools > Convert Main Data Source…
3. Enter the absolute url to your document library (e.g. http://yoursitecollection/subsite/your doc library name), and click Next.
4. Finish the wizard and view your Data Source. Expand the nodes and you should see the extra list column appear now.

This is also a great way to update the DIP on your List Content Type, especially if you have locally overridden the Optional/Required/Hidden settings on columns from the parent Site Content Type.

This workaround kind of defeats the whole point of setting the Panel at the Site Content Type level, but that's the only way I could get it to work.

I had the need to set Business Data and other column values in the ItemAdding event handler. I had to use ItemAdding, because I needed the changes committed so that the values I set would be visible on the Edit Properties screen immediately after uploading a document. Using the synchronous handler guaranteed that the values would be set.

I ran into the following issues with ItemAdding/ItemAdded event handlers:

• Setting AfterProperties for some columns was not working using the InternalName of the field.
• For Microsoft Office document file types, any data that I set in ItemAdding event via AfterProperties was lost after the first time the item was edited.

According to other people's experience, when using the AfterProperties in ItemAdding event, you must use the Internal Name of the field, not the display name.

properties.AfterProperties[list.Fields["Display Name"].InternalName] = somevalue;

When I did this, however, my experience was exactly the opposite. It was only by using the Display Name of the field that I could actually get any of the data to stick. I'm not sure if it was my particular mix of SP1, Site Content types, 64bit server, and the use of a Business Data column on the document library that was making it not work, but I simply had to use the display name, or else none of the data I set would stick (my column names had a mix of spaces and no spaces, and interestingly the column names with no spaces worked either way).

This whole architecture of hashtables of properties and trying to match up values seems to me to be pretty fragile, and I'm not sure why Microsoft went this way. It also seemed to create my second problem, where data I did set disappeared.

Once I got things working with the Display Name of the field, I then noticed that I had a problem with the data disappearing after I would edit the properties of the document. But this was only for Microsoft Office document types, things were fine for image files, text files, xml files. In debugging, I checked the values at ItemAdding, ItemAdded, ItemUpdating, and ItemUpdated. Everything looked as it should through ItemUpdating, but somewhere in between ItemUpdating and ItemUpdated, the data was lost, and my values were set to empty strings. It seems there is something strange with the parser for Office file types.

To get around this, I had to set the values again in the ItemAdded event and Update the ListItem.

I would have simply switched to using ItemAdded exclusively, since that seemed to retain the data in all cases, but my code was executing a call to the Business Data Catalog, and so was a little slow. This would cause errors from SharePoint when a user would change data in the Edit Properties screen after upload, because it would complain that the file was recently updated. Generally a problem I would expect from an asynchronous handler.

To solve this, I do the heavy lifting code in ItemAdding, and then simply do a resetting of the value in ItemAdded to ensure that it sticks.

properties.ListItem["DisplayName"] = properties.AfterProperties["DisplayName"];

properties.ListItem.SystemUpdate();

I still don't feel comfortable using ItemAdded for this, but it seems to be fast enough that I don't have any problems.

I recently had the need to create a custom SharePoint application page (living in_layouts directory) that needed to display the BDC entity picker control, so that a user filling out this form could select a BDC entity instance, exactly like a user would in a business data column defined on a list. It took a while to figure it out, here are the steps to get it working:

Step 1 – Dissecting the ItemPicker control

The control that MOSS uses to render the BDC entity picker is called ItemPicker. You can add it to your aspx page by adding the following reference and code:

<%@ Register TagPrefix="Portal" Namespace="Microsoft.SharePoint.Portal.WebControls" Assembly="Microsoft.SharePoint.Portal, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" %>

<Portal:ItemPicker class="ms-input" ID="YourEntityPickerID" runat="server" />

Like the People Picker control (PeopleEditor), you have properties that you can set, such as AllowEmpty, MultiSelect, NoMatchesText, etc. However, there is one more property that you have to set, ExtendedData, that cannot be easily set in the aspx markup, so I create a handler in the markup for the Init event, and then set all the properties in code behind:

<Portal:ItemPicker class="ms-input" OnInit="YourEntityPickerID_Init" ID="YourEntityPickerID" runat="server" />

Step 2 – Setting ExtendedData

In order for the control to work properly, you have to set the ExtendedData property, using the ItemPickerExtendedData Class. This needs to be done via code, so be sure to create a reference to your control first, and handle the Init event of the control:

using Portal = Microsoft.SharePoint.Portal.WebControls;

protected Portal.ItemPicker YourEntityPickerID;

protected void YourEntityPickerID_Init(object sender, EventArgs e)

{

    // Set extended data

    YourEntityPickerID.ExtendedData = GetExtendedData(

        Settings.Default.LOBInstanceName, 

        Settings.Default.EntityName, 

        Settings.Default.EntityDisplayNameColumn);

    // Set other properties

    YourEntityPickerID.AllowTypeIn = false;

    YourEntityPickerID.AllowEmpty = false;

    YourEntityPickerID.AutoPostBack = false;

    YourEntityPickerID.MultiSelect = false;

}

I've broken it out into a little helper method that takes in three parameters, the LOB Instance Name (name of your BDC LOB application instance), Entity name (the name of your Entity, e.g. "Customer"), and the entity display name column (the Name attribute in the TypeDescriptor of the column that acts as the display name column for your entity).

using Microsoft.Office.Server;

using Microsoft.Office.Server.ApplicationRegistry;

using Microsoft.Office.Server.ApplicationRegistry.MetadataModel;

using Microsoft.Office.Server.ApplicationRegistry.Runtime;

private Portal.ItemPickerExtendedData GetExtendedData(string lobInstanceName, string entityName, string titleFieldName)

{

    // Create a new ExtendedData object

    Portal.ItemPickerExtendedData data = new Portal.ItemPickerExtendedData();

    // Get the LOB Instance

    LobSystemInstance lob = ApplicationRegistry.GetLobSystemInstanceByName(lobInstanceName);

    data.SystemInstanceId = lob.Id;

    // Get the entity

    Entity entity = lob.GetEntities()[entityName];

    data.EntityId = entity.Id;

    // Set the primary column id (the id of the "Title" field)

    FieldCollection fields = entity.GetSpecificFinderView().Fields;

    List<uint> secondaryColumnIds = new List<uint>();

    foreach (Field field in fields)

    {

        if (string.Equals(field.Name, titleFieldName, StringComparison.OrdinalIgnoreCase))

        {

            data.PrimaryColumnId = field.TypeDescriptor.Id;

        }

        else

        {

            secondaryColumnIds.Add(field.TypeDescriptor.Id);

        }

    }

    data.SecondaryColumnsIds = secondaryColumnIds.ToArray();

    return data;

}

The important things happening here are:

1. Setting the SystemInstanceId (this marries the control to a particular BDC application).
2. Setting the EntityId (this marries the control to a particular entity).
3. Setting the PrimaryColumnId (points the control to the TypeDescriptor that acts as the Identity column)
4. Setting the SecondaryColumnIds (points the control to the TypeDescriptors for all the other columns you need to bring in).

Step 3 – Consuming the Data

To consume the data, perform some validation, and then try to get a PickerEntity object out of the Entities property of the control:

YourEntityPickerId.Validate();

if (YourEntityPickerId.Entities.Count <= 0)

{

    SPUtility.TransferToErrorPage("You must pick an Entity.");

}

PickerEntity bdcEntity = (PickerEntity)YourEntityPickerId.Entities[0];

if (bdcEntity == null || !bdcEntity.IsResolved || bdcEntity.EntityData == null)

{

    SPUtility.TransferToErrorPage("You must pick an Entity.");

}

Once you have a PickerEntity, you can get the ID of the entity instance via the Key property, the display name via the DisplayText property, and also get all the other column data via the EntityData property.

Note that the .Key property is the encoded ID of the entity instance (you can use EntityInstanceIdEncoder to encode/decode this ID). Note also that the EntityData comes back as a HashTable of TypeDescriptor ID/value pairs, so you can't index into this with a friendly column name. You'll have to traverse the Entity and its fields, get the TypeDescriptor ID of the column you are interested in, and then use that to index into the hash table. That can be a pain, so it's almost easier to simply grab the .Key, decode it, and then call FindSpecific() on your Entity.

Recently, a client had a requirement to filter data on the MySite profile page (person.aspx) based on the user whose profile page you were viewing. The particular scenario the client wanted was when user Jim was looking at user Mary's profile page, Jim should see some data (from the BDC) that was filtered to only include items that Mary was involved with. The challenge here was to determine which user's profile the current browsing user was looking at, pull the UserProfile data for that profile, and then provide that data as a filter for other web parts via connections.

It turns out this was quite easy.

Part 1 – Deconstructing person.aspx

The first step was to figure out how SharePoint internally knows which user you are looking at, via the querystring parameters. Person.aspx can take in several different querystring parameters to figure out which user it should display. Following are the available querystring parameters that it can take in (listed in the same order that the page checks for):

I figured that there must be some control on that page that reads those querystring values to figure out the user. I also figured that it probably is storing that information in the HttpContext.Items collection, so that it can be leveraged by all the web parts on that page that need access to the profile information.

I cracked open Person.aspx to see what controls are being used, and zeroed in on ProfileViewer (the control that displays Profile Property values).

   1: <SPSWC:ProfileViewer id="ProfileViewer" ShowBusinessCardIfEmpty="false" runat="server" />

I looked up ProfileViewer in Reflector, saw that it inherited from ProfileUI, and drilled into that object. Looking in InitControl, I found this code:

   1: this.objLoader = ProfilePropertyLoader.FindLoader(this.Page);

Bingo, a static method call to get a loaded profile. Nice. Drilling into the ProfilePropertyLoader, you discover that it is kind of a quasi-singleton, that will create a new object if it is not found in the HttpContext.Items collection, or return one that it already finds. The ProfilePropertyLoader has two properties that return a UserProfile object, ProfileCurrentUser (the profile for the browsing user) and ProfileLoaded (the profile of the user to be displayed, which may be the same as ProfileCurrentUser).

Part 2 – Building the filter web part

Now that I knew how to get the UserProfile object via ProfilePropertyLoader, I set about creating a filter web part, so that I could pass in filter or query values from the UserProfile to other web parts via Connections. MSDN has a great article on creating a Filter provider web part, so I followed that and added my code to get user profile values and pass those into other web parts.

   1: public ReadOnlyCollection<string> ParameterValues

   2: {

   3:     get 

   4:     { 

   5:         // Get a reference to the ProfilePropertyLoader, 

   6:         // which contains the reference to the UserProfile

   7:         Microsoft.SharePoint.Portal.WebControls.ProfilePropertyLoader loader = Microsoft.SharePoint.Portal.WebControls.ProfilePropertyLoader.FindLoader(this.Page);

   8:  

   9:         if (loader == null)

  10:         {

  11:             return null;

  12:         }

  13:         

  14:         Microsoft.Office.Server.UserProfiles.UserProfile profile = loader.ProfileLoaded;

  15:  

  16:         if (profile == null)

  17:         {

  18:             return null;

  19:         }

  20:  

  21:         string thirdPartyAppUserID = (string)profile["ThirdPartyAppUserID"].Value;

  22:  

  23:         if (string.IsNullOrEmpty(thirdPartyAppUserID))

  24:         {

  25:             return null;

  26:         }

  27:         List<string> thirdPartyAppUserIDList = new List<string>();

  28:         thirdPartyAppUserIDList.Add(thirdPartyAppUserID);

  29:         return new ReadOnlyCollection<string>(thirdPartyAppUserIDList);

  30:  

  31:     }

  32: }

Using this approach, I was able to pass in a filter value from a UserProfile property to BDC List web parts.

We've implemented Ontolica search (for MOSS 2007) for several of our customers. The product fits our service offerings very well, and provides a great value add for minimal cost and effort (compared with other search replacement products).

The sales team has been very helpful and responsive. They've provided trial licenses quickly and without hassle, have provided training and demos/overviews of the product, and have even been excellent interfaces to help escalate support incidents. I can't really say enough about the sales team.

I can't say that about their support however. So far, they have had just about the worst level of support I have experienced with a third party SharePoint product. If you file a support incident through their web site or via email, you are guaranteed to not get a response.

I filed one particular support incident, never heard back from anyone, until 2 months later, when I received an automated email asking me if my incident had been resolved. Another incident received no response until I involved the sales persons, who managed to look into the issue themselves. A co-worker filed another incident, and never heard a response. I'm not even convinced they have a support team.

Outside of the support, if you are interested in their product, keep in mind that they have very little extensibility. This can be important if you ever need to write any customized search code, and would like to take advantage of the Ontolica features in your custom search solution.

One client of ours had a need to run searches using impersonation, so we wrote code to leverage the MOSS search web service, passing in the credentials of the impersonating account. We would have liked to have been able to pass our returned results into the Ontolica web parts (or simply have been able to use Ontolica with impersonation), but the web parts all use internal hidden search objects that are obfuscated and cannot be interacted with (unless you like deciphering obfuscated code). This made it so that we had rich search tabs with all the Ontolica features, and then our customized search tab with stripped down functionality. To be fair, my beef also lies with Microsoft for closing up their own search with the SearchResultHiddenObject, which cannot be used easily without reflection and a big level of effort. I would have thought that a product that provides such a good experience with installation and configuration would have put more thought into extensibility (and support).

Overall I would give Ontolica 3 out of 5 stars.

If you've ever tried to add a named anchor hyperlink in a MOSS/SharePoint Wiki page, you'll find that the SharePoint rich text editor does not support adding named anchors (hyperlinks to areas in the same HTML document) via the toolbar buttons (you'll get an error dialog). I was able to find one workaround here, but it didn't quite suit my needs. Following is possibly an easier workaround using root relative urls. To get this functionality, you first need to switch to source view and add some html code by hand.

Add Anchors

Click the source button to switch to HTML source view. Add your anchor wherever you need it in the source, for example:

<a id=”nameofanchor” name=”nameofanchor”></a>

When done adding anchors, click OK to go back to the rich text view.

Linking to Anchors

To create links to the anchors you just created, you can do this easily from the rich text toolbar by clicking on the Hyperlink button.

In the Hyperlink dialog, for the Address, specify the root relative URL for the page, in the following format:

/firstsubsite/secondsubsite/wiki page library name/Name of Wiki Page.aspx#nameofanchor

You can get this url by going out of edit mode on the page, and back to viewing the page in regular view mode.

1. Copy the URL from the address bar
2. Remove the first domain name part of the URL (e.g. http://domain.com)
3. Add the named anchor portion (e.g. #nameofanchor) to the very end of the url.

Just for laughs, I thought I'd post this nice error message from SharePoint. Love the slash n's in there.

Overview

The current migration project I am working on is migrating a very large, 8 year-old portal and all its contents from an archaic BroadVision platform to MOSS 2007. BroadVision is absolutely one of the worst portal software packages I have ever seen, and is proving to be quite difficult to migrate into MOSS 2007. This is part 1 of an ongoing series of stories about this project and its challenges.

Migrating Forms

BroadVision has support for a crude version of custom web forms, that can be used to collect data and either email the data or save it in a database. These forms are standard html forms with html form tags, and almost all of them have a spot at the top of the form where some introductory html is placed to instruct the user on how to use the form, the form's purpose, etc. This project has about 200 of these forms, for almost every department in the organization.  These forms have very little workflow, with just some primitive options to email or CC people when new forms are submitted. There is no easy way to automate this migration.

Below is an example of one of these forms, which is a form to update an employee's address:

InfoPath vs Custom Lists vs ASP.NET Web Applications

The first decision to be made in migrating these forms is choosing the right technology. I presented 3 options to the customer:

• InfoPath – Pros: Flexible, supports validation, custom actions and workflows, can save data to sharepoint lists, libraries, and external databases. Cons: Requires training and design skills, and sometimes coding and development efforts
• Custom SharePoint Lists – Pros: Anyone can do it, easy repository for data, built-in export to Excel, Datasheet view, MOSS 2007 workflows. Cons: Limited validation support, little or no design control, data can only be placed in SharePoint
• Custom asp.net web application – Pros: Complete control over design and storage. Cons: Requires development effort

With over 200 forms to migrate, very little resources, plenty of new forms yet to be created, and lots of forms that need updating or reviews, the customer wanted every department to own, migrate, and create their own forms. The best solution in this case was the one that empowered the end users to create and manage their own forms as easily as possible.

For 90% of the forms, the custom list was the right approach, unless a form needed advanced features like validation, pre-population, etc. The only stumbling block then was having the ability to edit the form that is displayed when you create a new item (NewForm.aspx).

Introductory Text and HTML

Every form in the old portal had a space at the top where introductory HTML could be placed to provide instructions. Since a custom SharePoint list uses the default NewForm.aspx page when adding a new item, there is really no place where this intro HTML could be placed. To solve this, we used a special querystring to put the NewForm.aspx page into edit mode, where we could add a Content Editor Web Part above the form's contents.

Following is the method for getting this to work:

1. Go to the custom list and click the menus to add a new item.
2. On the New Item form, add a querystring parameter to the end of the url, and hit enter. For example, a typical New Item form's url might be:

   http://intranet/Docs/Lists/Tasks/NewForm.aspx?RootFolder=%2fDocs%2fLists%2fTasks&Source=http%3a%2f%%2fintranet%2fDocs%2fLists%2fTasks%2fAllItems.aspx

   Add the following text to the very end of the url: &ToolPaneView=2

   The new URL becomes:

   http://intranet/Docs/Lists/Tasks/NewForm.aspx?RootFolder=%2fDocs%2fLists%2fTasks&Source=http%3a%2f%2fintranet%2fDocs%2fLists%2fTasks%2fAllItems.aspx&ToolPaneView=2

3. This puts the page into Edit mode, and displays the Add Web Parts toolpane. Click Next until you see the Content Editor Web Part. Drag it into the Web Part Zone at the top of the page.

   

4. In the newly added part, click the link to open the Tool Pane ("To add content, open the tool pane and then click Rich Text Editor.") , and paste your introductory text into the Rich Text Editor. When done, click OK, your instructions are now live on the Form.

Granted, telling an end user to hack a querystring to add a web part is not the most user friendly approach, but it is certainly something that can be documented well, and is certainly easier than InfoPath training. Of course this step could also be done easily with SharePoint Designer, but that is another layer of complexity that the users don't need.