I’ve jut finished reading the biography of Steve Jobs by Walter Isaacson. It’s a good read about the man who built and rebuilt Apple.

This post won’t try for any deep insights, no “he was a brilliant but flawed individual, blah blah blah”. It’s merely intended as a small list of key lessons from a successful dude.

• A closed system may be a better enabler for creativity, connected experiences and a better all-around product for users
• A players don’t like to work with B or C players. Only hire A players and get rid of the rest.
• Face to face interaction and tight co-location can lead to serendipitous discoveries
• integrated operations are very important in helping the company operate as a unit. Pillars block teamwork. Avoid the mistake Sony made with the iPod.
• you can and should have vigorous arguments with people
• don’t be afraid of starting over, of throwing away something that is stopping your product from being great
• there is no such thing as a single person having all the ideas.
• give credit, people hate having their ideas stolen or credit taken for them
• you aren’t Steve jobs. You can’t get away with shouting at people.
• being honest gets you a better outcome
• don’t forget or ignore your family
• better to be the one who breaks the model and leads from the front than the one who plays catch up
• creativity is a discipline too. It takes hard work.
• Users appreciate a great experience.
• Simplify ruthlessly
• You must have end-to-end elegance in design.
• You get great value in automation and automated logistics (e.g. iPods have 2 days of stock, cut down from 3 months) which saves them millions
• the boss can be wrong and needs you to argue with him/her

There may be others, but that’s a decent list of learning points.

I’ve recently passed my TOGAF9.1 Certified exam and wanted to share my experience of preparing for the Foundation and Certified exams.

Overview

TOGAF (The Open Group Architecture Framework) is a Framework for Enterprise Architecture. To become TOGAF Certified, you need to pass two exams, Foundation and Certified.

• Foundation is simpler, 40 multiple-choice questions; requires 55% to pass.
• Certified is much more difficult with only eight questions, but they require the review of a case study and it’s open book. 60% is enough to pass the Certified exam.
• You must pass both, in sequence.
• You can do both on the same day
• You can test at Prometric Exam Centres
• They’re around £200 each so it’s worth studying to ensure you pass.

How TOGAF can benefit you

Getting certified just for the sake of it (or to boost your salary) is pretty pointless. I found that TOGAF helped my day-to-day engagement with Enterprise Architects and it’s allowed me to add value for clients by bringing TOGAF concepts or techniques to bear. It can greatly expand your toolbox and act as a great source of reference material.

A few specific examples:

• I was able to use the Architecture Review process and check-list approach to create a review standard where none previously existed for one client
• I used the Risk Management framework to catalogue and prioritize architecture risks

Study Materials

I bought the full study guide from the Open Group website. It cost around $60usd and for you money you get Foundation and Certified study guides.

You will also want to familiarize yourself with the full online TOGAF reference. It’s dry stuff to begin with, but I’ve found that as I become more familiar with it, it has become incredibly useful. You can buy this, but I don’t see why you would.

Foundation is rote memorization so I used a number of techniques to aid memorization.

• Created vocabulary lists on Quizlet
• Created practice questions in this downloadable excel (can be very useful unless you setup your own Moodle server, else a bit difficult to use)
• Used Flashcard software (Repetitions for iPhone) to bang lists and concepts into my head. You can download my flashcards but you’ll need repetitions of Mnemosyne to use them.

All three were useful and I’m not sure I would  have passed without doing them. I found that I could not stare at the study guide and absorb the content, so using active tools helped hammer the content home.

For certified, I did the various practice questions (study pack and the few ones on the Internet).

I definitely studied far more than I needed to pass the exams, but I didn’t do TOGAF just for the certification. If you just want to pass, you can probably get away without the Flashcard stuff.

The exams

If you get the self-study guide, the exams are pretty much identical in nature to the included practice tests. I won’t / can’t talk about the questions themselves. If you can pass the practice test for foundation comfortably, you’ll probably be fine for the actual exam.

For Certified, I found myself wanting more practice questions but there aren’t many (legit ones anyways). What I did was create a process for answering questions and followed this rigorously.

The question format of questions in the study guide is pretty much:

1. Background
2. Problem / Concern
3. Question: What is the best approach to X according to TOGAF? (or similar)

My approach

1. Write down the requirements that must be satisfied (usually more than one)
2. Write down what TOGAF says about X (usually between 5-10 things)
3. Write down the 4-5 suggestions aligned to each answer A,B,C,D
4. Match to see which of A,B,C,D best meets the concerns AND is aligned to TOGAF

I passed comfortably. If you’ve read the study guide and understand the concepts, this approach is pretty fail safe.

Exams are done at Prometric test centres. The Certified exam is open book and so includes a PDF reader inside the exam. The reader is not as good as Adobe PDF, but it’s good enough.

You can find links to sample exam questions below

• Free (and legit) practice exams
• Sample questions from Udayan Banerjee
• Sample questions from Chris Eaton

If you’re getting the irritating “We couldn’t find what you’re looking for” error, it can be frustrating to fix.

Ultimately I had to fix the problem from outside of Outlook. I went to my “Change how Windows Searches” option dialog and rebuilt the index. I tried the exact same thing from inside Outlook but nothing ever happened.

Important: Before getting this to work, I also removed Outlook as an indexed location, restarted my PC, and then re-added it. However, this only partially helped, and there was still stuff missing from the index.

This is as much for me as for anybody else, but I hope visitors to this page find it useful.

Obviously, you can always try SharePoint’s standard verbose logging, but I’ve never found that to be terribly useful for search.

Debugging the Crawler

You might consider trying to use Fiddler2, the same way you would with SharePoint 2010. SharePoint Blues has a good overview on how to turn this on. The steps for 2013 will obviously be different.

Debugging Pipeline Stages

No entries yet…

Debugging Custom Pipeline Stages

No entries yet, but since they’re web services you can write on your own, debugging them shouldn’t be intractable.

Debugging the Index

There’s an interesting post on the MSDN forums that is worth a read. It indicates there is no more FIXML, and that the index is stored in a binary format. Microsoft is apparently creating a tool to inspect the index but who knows how long that might take.

Debugging Queries

SP2013 Search Tool

There’s a query tool written by somebody at Microsoft which looks quite handy and is compatible with SharePoint Online as well. Ideal when you can’t login to the server. You can get it from CodePlex. This looks very similar to the MossMan query tool in SharePoint 2013.

Use it when:

You can’t directly login to the server.

You want to see what the index knows about a specific document.

What it won’t tell you:

What specifically the query server sent back to a specific query submitted via the browser.

Developer Dashboard

There’s a suggestion in this deck by Agnes Molnar (SharePoint MVP) that you can use the developer dashboard, but there’s no more detail than that. Looks promising though, at least in a pre-production environment.

Debugging Search with SharePoint Online

•  Look at the SP2013 Search Tool mentioned under Debugging Queries

Using Quizlet I’ve created a free online quiz for TOGAF definitions. You can use them to study and even create tests for yourself. It’s a handy way to work through the 70+ definitions. I find it more effective than simply staring at the book and I hope you do too!

Do you need to know all the definitions for the exam? Of course not, but it helps point you to areas you might be a little weak.

It’s also free. I hope you find it useful, please let me know if you do.

There’s a defect whereby your search results in SharePoint will not have document icons if you’re using FAST for SharePoint.

This will happen to you, whenever you’re using a Document Library created from a custom Document Library template (i.e. Type != 101).

Why this happens

The problem is that the contentclass of documents from custom libraries indexed by FAST in is not compatible with the way the Core Results Web Part determines which icon to display. Under normal circumstances, FAST allocates documents a contentclass of STS_ListItem_DocumentLibrary and when search results are returned to the Core Results Web Part as an xml document, our document is accompanied by an imageurl element which points to an icon within the layouts folder (i.e.  /_layouts/images/icdocx.png).

When using a custom document library, let’s say with a type of 15000, the contentclass will be STS_ListItem_15000 and the Core Results Web Part will be returned an imageurl element of /_layouts/images/STS_ListItem16.gif.

This has been accepted as a bug by the product team. Unfortunately, So far as I know, it is not likely to be resolved in either upcoming hotfixes or Service Pack 1 (I have no visibility of any future CUs or Service Packs). The Microsoft-recommended workaround is to create a Custom Pipeline Extension whereby library items that have a contentclass of STS_ListIem_<a number> are converted to STS_ListItem_DocumentLibrary.

I’m checking into whether I can post some PSS-provided sample code for the pipeline extension, but until then, visit this page on MSDN to get instructions on how to create one.

Update 21 Sept 2012:  Baastian Kortenbout has provided code which addresses the issue. This can be downloaded here: FS4SP2010PipeLineExtensionIconFixer. I make no guarantees about the code, you assume all risks and responsibility should you chose to use it in any way, blah blah blah.

• QR Server debugging queries >> Use this to inspect queries coming from SharePoint to FAST.
• FFD Dumper for inspecting crawled properties >> Use this to inspect crawled properties
• MossMan FAST Query tool >> A handy little tool which allows you to build queries yourself, and interrogate FAST. Useful when writing custom FAST control.
• FASTSearch2010.chm >> A downloadable MSDN reference which holds a ton of PowerShell commands that relate to configuring FAST
• Information Worker Demo VM >> Downloadable VM from Microsoft that is setup and ready to use. Comes with FAST 4 SharePoint preconfigured
• PowerGui >> IDE for working with PowerShell. Invaluable if you’re working with PowerShell for a longer period of time.
• FAST Query Logger >> A new tool which makes working with the QR Server significantly easier.

Previously I blogged about the QR Server, my first port of call whenever I need to debug FAST. At that stage I ask the question “What does FAST know about this document?” and the QR Server is the best place to start looking.

Sometimes, the information in the QR Server isn’t enough to figure out what’s going wrong. If that’s the case, my next question is usually “What is FAST told about this document?”

As part of this process it is critical to understand the difference between Crawled & Managed Properties.

A Crawled Property is something that is fed into the FAST document processing engine. It may or may not decide to index and keep what it finds. If the information it does is not ignored it will end up in the Full Text index, a Managed Property, or both. Crawled Properties à What FAST is told, Managed Properties à What FAST knows and has decided to keep.

While the QR Server is for managed properties, you use a tool called FFDDumper to inspect crawled properties.

The debugging process for using the FFDDumper goes like this:

1. Enable the FFDDumper & reset your document processors
2. Mark an individual document for being re-crawled
3. Start an incremental crawl
4. Inspect the generated FFD files
5. Turn off the FFDDumper and reset your document processors

Enabling the FFDDump

There is a file in your FAST installation folder called optionalprocessing.xml (found under %FASTSEARCH%\etc\config_data\DocumentProcessor\). This file is for configuring optional processing pipelines (duh!).

Find the element that looks like the one below and set it to yes.

<processor name=”FFDDumper” active=”yes” />

THIS WILL SLOW DOWN YOUR SERVER AND GOBBLE DISK SPACE! Use this for temporary debugging only, and really, only use it for incremental crawls or else the sheer volume of data will make it very difficult to find what you want.

Reset your processing pipelines from the FAST PowerShell console with a

> psctrl reset

Marking a document for re-crawl

In SharePoint navigate to your FAST Content Service Application and mark a single document for a re-crawl. Alternatively, make a small change to a document (make sure the document is published).

Kick off an incremental crawl by going to the context menu of your Content Source.

Inspecting the generated FFD Files

If you navigate to %FASTSEARCH%\data\ffd you’ll find one or more folders. Go to the newest folder and look inside. There should be a few .ffd files, one of which will contain the ssic of your document (you can get this from the QR server).

Each line from the FFDDumper will look something like the following:

The format of each line includes:

• Length of property identifier & name
• Property identifier (may be a GUID)
• Property name
• Variant Type
• Length of the property value (often preceded by an ‘s’)
• The property value

Only lines with GUIDs are available to be mapped into Managed Properties. Lines without GUIDs are internal to FAST and are not accessible. If you look at the screenshot below, the unfortunate implication is that you can’t write your own pipeline extension to run against the raw crawled data.

Reset the FFDDump

Once you’re done, reset the FFD Dump to its regular configuration

<processor name=”FFDDumper” active=”no” />

Finally, run a psctrl reset to load the updated settings of optionalprocessing.xml

I’ve recently created a new SharePoint site collection but was missing the FAST features out of the Site Collection Settings. I had a site collection within the same Web Application where the features were present and so was very confused. I already had added myself to the FASTSearchAdministrators and the FASTSearchKeywordAdministrators groups on my FAST server which can also keep the FAST options hidden, and besides, I already had them visible elsewhere.

There were no obvious features to activate within the Site Collection features, and I had standard, enterprise & publishing features enabled. Noting the url of one of the pages in the working site collection I tried loading it up in the broken one. For example:

• site/workingSC/_layouts/contextualkeywordmanagement.aspx
• site/brokenSC/_layouts/contextualkeywordmanagement.aspx

At this point I got the standard SharePoint error popup with the message FAST Search site collection features are not activated. Pretty obvious what’s going on, the feature isn’t activated and there’ s no obvious way to do so via the UI.

Finding the Feature GUID

Digging through the API using Reflector reveals this method which any reasonable developer can infer will give you the error message above.

private static void ValidatePremiumSiteCollectionFeature()
{
   if (SPContext.Current.Site.Features[new Guid(SearchAdminConstants.FAST_SITE_ADMIN_FEATURE_ID)] == null)
   {
     ReportError(StringResourceManager.GetString(LocStringId.FSAdmin_Fast_Site_Admin_Feature_Not_Activated));
   }
}

A little more use of Reflector gets you the GUID for the FAST Site Admin Feature (5EAC763D-FBF5-4d6f-A76B-EDED7DD7B0A5).

Armed with this guid you can get the Feature definition. You don’t need it, but it’s interesting regardless (I’ve never seen this available via the UI).

Get-SPFeature | ? { $_.id -eq “5EAC763D-FBF5-4d6f-A76B-EDED7DD7B0A5” }

DisplayName                    Id                                       Scope                         
-----------                    --                                       -----                         
SearchExtensions               5eac763d-fbf5-4d6f-a76b-eded7dd7b0a5     Site

From here, if you want to add the feature to a site collection simply run the following PowerShell

$siteWithNoFastFeatures = get-SPSite http://mysite/noFAST
$siteWithNoFastFeatures.Features.Add(“5EAC763D-FBF5-4d6f-A76B-EDED7DD7B0A5”)