Custom Ranking Models with SharePoint 2010: Background, Value and Administrative Overview

Posted on June 21, 2010 by

5


Introduction

At a recent SharePoint user group session, where I spoke on Enterprise Search in SharePoint 2010, I was introduced as discussing the “black art” of search. The reason, I think, that search was described in that way reflects the way in which most people think about search: something that is pervasive and quickly yields the information required, usually on the first page of results. It’s fair to say that the average user doesn’t comprehend the complexity and effort required to maintain relevancy, and nor should they need to.

With the accuracy, power and flexibility of the big-name internet search engines comes a certain expectation from end users in terms of search engine performance behind the corporate firewall. Striving to achieve the same level of fidelity of search results behind the firewall with most Enterprise Content Management (ECM) systems presents a major challenge and administrative headache. SharePoint 2010 alleviates some of this pain and provides the ability to override the default ranking model for enterprise search results. This enables the development of application and organisation specific enterprise search applications which provide contextual results.

This article is the first of a series I’ll produce on custom ranking models with SharePoint 2010. The aim of this article is to provide a background on custom ranking models, why they’re important and how SharePoint 2010 supports the implementation and management of custom ranking models through PowerShell and XML configuration files.

Background on Custom Ranking Models and SharePoint 2010

The ranking model that SharePoint 2010 uses is based upon BM25F. BM25F is actually BM25, a ranking function originally implemented at London City University, with field weighting and per-field length normalisation and provides support for weightings for managed properties/metadata amongst other things. It is at this point that I would strongly urge the reader to investigate the references at the end of this article. There is an extent to which the successful implementation of custom ranking models depends upon foundation knowledge of BM25F.

There are nine ranking models available out of the box with SharePoint 2010 and these are:

  • Main Results Default Ranking Model
  • Expertise Social Distance Ranking Model
  • High Proximity Ranking Model
  • Main People Social Distance Model
  • Expertise Model
  • Name Social Distance Ranking Model
  • Name Model
  • Main People Model
  • No Proximity Model

There is no support to modify these ranking models and you must create your own. For reference purposes, these can be viewed within the MSSRankingModels table of the SSA Admin DB as shown below:

I would urge you to ‘look’ at these ranking models in a development environment because that will give you an idea of the power and flexibility that you can leverage through XML files alone.

Why are custom ranking models important?

Custom Ranking Models are important because they allow search results to be tailored based upon what we know is already important within our organisation. Consider an Enterprise Search Center for a Sales division within an organisation. Sales people are sales driven and have an much greater interest in specific content types and information than others e.g. a PowerPoint slide deck relating to a SharePoint sales pitch rather than an extensive Word document along the technical aspects of SharePoint Enterprise Search. Ranking Models, simply put, allow us to define what is appropriate and relevant within a specific organisation and promote/demote search results based on the criteria that we define.

The Ranking Model Schema

Microsoft provide a ranking model schema, given below, which provides a blueprint on which to validate your custom models.

<rankingModel name=”string” id=”GUID” description=”string” xmlns=”http://schemas.microsoft.com/office/2009/rankingModel”&gt;

<queryDependentFeatures>

<queryDependentFeature pid=”PID” name=”string” weight=”weightValue” lengthNormalization=”lengthNormalizationSetting” />

</queryDependentFeatures>

<queryIndependentFeatures>

<categoryFeature pid=”PID” default=”defaultValue” name=”string”>

<category value=”categoryValue” name=”string” weight=”weightValue” />

</categoryFeature>

<languageFeature pid=”PID” name=”string” default=”defaultValue” weight=”weightValue” />

<queryIndependentFeature pid=”PID” name=”string” default=”defaultValue” weight=”weightValue”>

<transformRational k=”value” />

<transformInvRational k=”value” />

<transformLinear max=”maxValue” />

</queryIndependentFeature>

</queryIndependentFeatures>

</rankingModel>

This basic blueprint will be explored in future articles as we discuss the actual development of custom ranking models. For now, I would draw your attention to the QueryDependantFeature and QueryIndependantFeature XML elements.

Query Dependant Features within the ranking model provide the ability to define managed properties which supports dynamic ranking. Query Independent Features are static ranking properties that apply irrespective of the query issued or the results returned. Why is this important? Well… we may want to stipulate that we always want to rank PowerPoint presentations above Word documents. We could then express this as a Query Independent Feature.

Administrative Overview

The power of custom ranking models becomes even more impressive when we realise that developer intervention is not necessary for their implementation and management. The necessary tools to implement and manage custom ranking models are PowerShell and XML.

Managing custom ranking models with PowerShell

There are four PowerShell cmdlets that provide the functionality to manage existing, and implement new, ranking models.

Get-SPEnterpriseSearchRankingModel Returns a ranking model
New-SPEnterpriseSearchRankingModel Adds a ranking model to a shared search application
Remove-SPEnterpriseSearchRankingModel Deletes a ranking model
Set-SPEnterpriseSearchRankingModel. Sets the properties of a ranking model for a shared search application

 

These speak for themselves and I won’t discuss each in turn any more than the descriptions given above. Suffice to say, I urge you to look at those cmdlets within PowerShell and take some time to understand their parameters.

Applying custom ranking models in Enterprise Search

Once you have a custom ranking model applied, using the aforementioned PowerShell cmdlets, there are three ways in which you can test the effects that other ranking models have on search results.

  1. Add rm={GUID} parameter to your search centre query string

     

    This is the easiest way to test the effects that a custom ranking model will have on search results and is probably the route you will take during development and testing. To quickly see how search results change based on different ranking models, you can take one of the GUIDs from the ModelId column in the MSSRankingModels table and apply that.

  2. Export Core Results WebPart, modify RankingModel parameter and reimport

     

    This is useful when developer intervention is not required or available. A standard out of the box Core Results Web Part can be exported to a .webpart file. An administrator is then able to modify this file and insert the GUID of the ranking model to use before reimporting the file into the page or Web Part gallery.

  3. Inherit the Core Results Web Part and wire up your ranking model in code

     

    This is the most likely option when there is already a development effort underway or there are other specific customisations to the Core Results Web Part needed.

 

What’s next?

The next article will explore the development of custom ranking models. This would benefit from an entire blog post of its own. For now, and before you embark on developing custom ranking models, I would urge you to begin to understand the fundamental concepts of BM25F and begin to identify what information is important to your organisation that you can use to influence ranking.

References

Microsoft Cambridge at TREC–14: Enterprise track
http://trec.nist.gov/pubs/trec14/papers/microsoft-cambridge.enterprise.pdf

About these ads