October 10
Using PowerShell to create new sites based on site-scoped WebTemplates

WebTemplates are definitely a powerful new construct in our SharePoint 2010 toolbox.  WebTemplates definitely come in handy as they can be deployed as sandbox-compatible features.

Creating a site based on a web template is pretty straightforward via the UI. Basically it just shows up as another site template option. As a user creating a site, you’d never the know the difference between a farm-scoped or site-scoped WebTemplate. However, if you want to use PowerShell, you will notice that your PS scripts will take on a slightly different shape based on how the WebTemplate is scoped.

If the WebTemplate is deployed as a farm-scoped feature, then you can easily use New-SPWeb in the following manner:

new-spweb $url –template “{GUID}#WebTemplateName”

 

where GUID represents the parent feature ID.

If the WebTemplate is deployed as a site-scoped feature, then your PowerShell needs to be adjusted.  Otherwise, if you attempt to use new-spweb, you’ll get the following error message: Template is not found and is not applied.  This effectively means the cmdlet could not locate a farm-level template to apply to the new site.

For example...

image

 

There are two ways to circumvent this problem:

  1. Once the site is created, call ApplyWebTemplate
  2. Before the site created, grab a reference to the appropriate WebTemplate and provide it as a value to the SPWebCollection.Add method on the parent site.

Examples

Calling ApplyWebTemplate

$url = "http://sitecollection/site1"

$w = new-spweb $url

$w.ApplyWebTemplate("{GUID}#WebTemplateName")

 

Calling SPWebCollection.Add

$url = "http://sitecollection"

$w = get-spweb $url

$template = $w.GetAvailableWebTemplates(1033) | ? { $_.name -eq "{GUID}#WebTemplateName" }

$w.Webs.Add("site1", "sample site 1", "sample description", 1033, $template, $false, $false)

 

The difference between the two methods basically boils down to the language selection for the new site. With the simple call to ApplyWebTemplate, the new site uses the same language as the parent. By grabbing the reference to the web template beforehand, you have more control.

-Maurice

October 07
SharePoint Conference 2011 Wrap Up

This year’s SharePoint Conference was probably one of the most interesting conferences that Microsoft has hosted in the past few years.  The attendance was solid and presentations covered the spectrum from 101 fundamentals over all the way over to nitty-gritty details.

Monday morning I had a chance to present on managing the Sandboxed Code Service (SP376).  I was a little skeptical that we’d fill a room with 700 seats, but I was very pleasantly surprised to see the room fill up before we switched on the microphone.  There were a ton of good questions afterwards as well.  Thanks to everyone who attended and posted all the great messages on Twitter. If you have any questions I was unable to answer, please feel free to reach out.

The conference was also a great chance to run into folks.  I saw many old friends – some that I haven’t seen literally in years – as well as many clients and former Critical Path Training admin and dev students.  It’s always amazing to see the positive energy!

The Aptillon team also had record number of presentations at the conference.  We had 7 presentations by 6 different teammates.  As a company partner, that is definitely very cool stat however the nicer fact is that we had a chance to hang out.  Since we’re spread out all over the US and constantly on the go, it’s rare to have more than 2 people in the same room at the same time. 

There was also a record number of Microsoft Certified Masters from across the globe at the conference.  How cool is that?  I remember the days when Spence and I were the only MCMs who weren’t employed by Microsoft. :) MSL also announced a new certification – Microsoft Certified Architect.  It’s really nice to see program growing!

Great conversations all around. New projects, new ideas, confirmation of design decisions... chatting about the sandbox, helping folks get a better perspectives on PowerPivot and it’s amazing potential, the cloud, getting out of the sandbox (aka azure), Windows Phone 7, watching cloud-servicing applications such as Sharevolution hit their stride, building new partnerships... truly exciting stuff!

-Maurice

September 27
Regex for Wiki Page regions

In an earlier post, I briefly touched on the value of creating your own InsertWebPartIntoWikiPage method. Part 1 – solved.  Now comes Part 2 – where am I injecting my web part?

That’s a valid question that is easily answered whenever you’re using the SharePoint wiki page editing tools within the browser.  The tools allow users to basically pick a spot in the rendered text, click on Add Web Part, and you are done.  The tools know how to safely inject your web part without forming bad html under the hood.

From a programmatic sense, this is much harder because we don’t get the “smarts” of knowing where to insert the web part markup.  Most users that try to use InsertWebPartIntoWikiPage invariably fail the first few times because they forget to account for the fact that they are injecting web part markup into an html blob and therefore need to properly account for well-formedness (is that a word?).

The net result is that folks will create code that looks like…

InsertWebPartIntoWikiPage (file, webpart, randomValueKeepFingersCrossedNothingBreaks)

The validity of the resulting HTML becomes something like rolling the dice at the casino.  Most folks can start with an empty page and count the number of characters to the first legitimate location where web part markup can be injected.  However, once the page is created and populated, all bets are off and IWPIWP becomes more of a hindrance than an aid but it doesn’t have to be that way.

Wouldn’t it be nice to be able to readily determine where the valid injection points are located? Sure! The answer comes in the form of good old fashioned regular expressions.

If you’re like me, you’ve got a nice little regex library for most any html operation (tag stripping, tag collection, validation, etc).  Well, here’s one more that you can add to the list… a regex to safely identify the your Wiki page regions!

imageThis little regex allows you to easily determine how many regions are in your Wiki page and, more importantly, where they begin and end.  The regex groups the valid editable regions with the name “InnerHtml”.  They represent the blue-bordered regions whenever you are editing a wiki page in the browser (as shown below).

image

In it’s simplest form, you can now inject your web parts in the right places without destroying your existing HTML.  For example, in the following clip, we add a web part at the start of the second region in the page.

image

Done! Page is updated with my web part in the right place and it all just works.

The really nice thing is that since we’re only identifying the HTML within the region, we can easily use the regex to update the text as needed in the region of my choice. 

Here’s another example where we identify a region and then replace it with new text.

image

What other tricks can you come up with?  How about identifying the web parts that exist in the page?  How about placing the new web parts in mid-stream w/o breaking the existing HTML structure?  All possible with regex, especially now that we’ve clearly identified the InnerHtml of your regions! 

You got to love the power of regular expressions because with a little ingenuity you can solve each one of those problems. 

-Maurice

August 09
Fixing the HTML that InsertWebPartIntoWikiPage generates

Buried deep in the recess of the WebPartPages class, there is a static method that belongs to the WikiEditPage class and its sole purpose is to allow you to insert WebParts into a wiki page.

The static method Microsoft.SharePoint.WebPartPages.WikiEditPage.InsertWebPartIntoWikiPage basically takes 3 parameters – the file, the webpart, and the text position of where you want to inject the web part.

All is fine and dandy except for one thing – the html that is produced by the method is invalid.  The method basically injects the html shown on the slide below. 

image

Take special note of the next to last <div> tag. 

Self-closing tag results in broken wiki page!

<div style='display:none' id="vid_{0}"/>

The tag as written is self-closing.  Unfortunately, when the text is processed by SharePoint’s parser, the self-closing notation is not recognized as valid html and the parser leaves this tag open.  This results in a very broken wiki page because the closing </div> does not exist for the segment of html that was just injected into the page.  Take a look at the image below – 3 start tags but only 2 end tags.

image

The fix is pretty easy.  Create your own version of InsertWebPartIntoWikiPage! 

All you need to do is change that next to last div tag from one that uses the self-closing notation over to one that properly uses an end tag.

image

Once you do this, your html segment will look like...

image

Nice and clean with the right number of start and end tags. More importantly, your wiki page will work as expected.

Later this week, I’ll show you a really cool way to use regex to control the location of the web parts that you’re trying to add to the wiki page.

-Maurice

Reference Post: Regex for Wiki Page regions

August 05
Options to using Import/Export

Earlier today, Todd Klindt and I had a brief twitter chat.  Basically, he ran into some problems with a site that had been run through the wringer with Import/Export.  Todd asked a really good question for which the answer requires a little more space than twitter allows.  :)

For those that are relatively new to SharePoint, they don’t often times don’t get a chance to learn what works or what needs to be avoided until it’s too late. Let’s take a brief look at the history of this feature set so that if you’re new to SharePoint, you’ll know where some issues may pop up.image

Import/Export is one of those features that needs to be used with caution.  Why?  Well, we have to go back to RTM of 2007 when the world was first introduced to Import/Export (aka PRIME).  It was a very broken toolset.  Fast forward a year or so, then the Infrastructure Updates for 2007 was released.  This was a special set of fixes designed to repair the underlying Import/Export engine.  The IU made the feature set much more usable; however, there were a core set of things Import/Export still could not do.  To the average SharePoint administrator, this basically means Import/Export is NOT a full fidelity solution for backing up and moving data.  Again fast forward to 2010 RTM.  Import/export is still pretty much unchanged.

At the site (aka web) level, Import/Export is without a doubt a very problematic feature. Import/export does not offer full fidelity data export. Thus, at the site level, you’re somewhat gambling with your data. Sometimes it will work and other times it may not. There are a lot of threads all over the place that help us see that Import/Export-ed sites tends to have weird and random behaviors.  As an administrator, the last thing I want to do is chase down problems with restored data.

Thought to ponder: Did you ever wonder why 2010’s SP1 added site Recycle Bin functionality?

Is Import/Export really that unreliable?  The answer is generally yes.  However, we definitely need to give Import/Export some credit. 

At the list and item level, Import/Export can actually go the full cycle. 

The more granular you are (i.e. items), the easier it is for Import/Export to actually work as we expect it to. Using Import/Export to move items is what is known as “cherry picking”.  It works and that leads us to the one of the possible options to Todd’s question...

“What (are the) other options for breaking up a site collection?”

Generally speaking you need to migrate your data.  You’ll either need to ...

  1. explore using Import/Export at the item level or
  2. work with a custom solution (written in C# or PowerShell) that extracts the data locally and pushes it to a new destination. 

Migration is not nearly as simple as just clicking on the Central Admin ui to export a site, then repositioning it.  However, it’s the only way you be assured that all the data you care about is where it needs to be and that your site is fully functional.

Should you build or buy the tools to migrate your data? It depends on your needs, skills, and timeframes.  There are lot of good tools – from free to pay.  PowerShell and custom code are definitely viable options.

Migration does take a little more planning but it inevitably makes for a much nicer end result.  It’s a great way to reduce clutter (e.g. no need to migrate stuff that doesn’t matter!) and you can rest easier when the process is done.  It’s worth calling out that with any migration plan, though, you should incorporate a testing strategy.  Testing will validate that your migration technique will successfully meet your requirements. 

Happy migrations!

-Maurice

July 11
Integrating post-processors in SharePoint builds

Have you ever needed to process an assembly prior to having it packaged up for deployment?  A classic example of this requirement is using an obfuscator or signing/watermarking tool.

Prior to the introduction of the VS2010 SharePoint tools, the general advice was to use AfterBuild (reference How to: Extend the Visual Studio Build Process). Unfortunately, this doesn't really work when dealing with solution packages built in VS2010.

The reason is pretty simple – AfterBuild is too late.  You can run your assemblies through the tooling process, but the modified assemblies won’t be a part of your solution package.  The packaging step picks up the assemblies before AfterBuild kicks off.

How about AfterCompile? This window of opportunity for post-processing your assembly is ok, but not entirely what you want to use.  You will get your processed assembly into the solution package; however, since the tool will modify your assembly, in the long run you will be wasting a lot of time and cpu as AfterCompile will be called multiple times.  On a fast running process, you may never even notice this is happening.  On a long running process, your mods will trigger multiple calls to compile and the time to completion will literally get out of hand.

The right solution?  Override AfterLayout.

This target is new target that is listed as a dependency by the CreatePackage target found in Microsoft.VisualStudio.SharePoint.targets.

 

image

 

All you need to do is open up your project file and add a definition for AfterLayout as shown below:

 

image

 

As you can see, AfterLayout is the perfect solution!

The really nice thing is you can use conditional expressions to define when to run the post-processing tool.  In the example above, the tool is run only when working with Release builds.

The build process will go through and copy all the files into the pkg folder.  AfterLayout is called once all the files are dropped into the folder, but before they are bundled up into the wsp.  Your post-processing tool can then reference the built assembly from the pkg folder.

 

image

 

Ta-da!  Done.  Your post-processing tool will run when you need it w/o taking the multi-run penalty of using AfterCompile

Just to give you an example of how efficient AfterLayout is compared to AfterCompile, we recently reduced our build times on one of projects from roughly 3 hours to under 32 minutes.  Old habits of defaulting to AfterCompile basically cost us a lot of time early on in our dev cycle... I can’t tell you how nice it was to get builds back down to normal times.  :)

Happy coding!

-Maurice

June 28
SPDeletedSite - quick overview of SP1’s improved recycle bin

With the release of SharePoint 2010 SP1, admins now have the opportunity to work with an improved recycle bin.  Sites and site collections can now be recovered if they have been deleted via the SharePoint browser interface. 

Site Recycle Bin

Service Pack 1 will introduce long awaited Site Recycle Bin functionality that enables self-service recovery of site collections and sites.   In the past IT Professionals were tasked with restoring entire databases to recover deleted site collections and sites and would generally require expensive restore environments to support the task.  Now in Service Pack 1 administrators can quickly and easily recover site collections and sites accidentally deleted by their owners in a process similar to that of the Recycle Bin we have for Lists, Libraries, and Documents.

(excerpt from SharePoint Team Blog)

How do you get started with recovering your deleted sites?

First, you have to upgrade your databases to SP1.  It is worth calling out this simple fact, because if you don’t you will get an error message that states “Specified method is not supported” when you try to delete a site (funny bug - you can no longer delete sites!). 

image

With your databases upgraded, the next step is to figure out what you want to undelete.  If you have deleted a site and the parent site collection is still up and running, all you need to do is jump over to the Site Collection Recycle Bin.  Your deleted site will appear in the second stage recycle bin (aka “Deleted from end user Recycle Bin”).

image

Pick the site you want to restore and you’re done.  Note that if you try to restore a child of site that has also been deleted, you need to restore the parent first.  In the image above, if you try to restore “/sites/Asta/Parks/GasWorks/sub1” before restoring the parent “/sites/Asta/Parks/GasWorks”, nothing happens.  In fact, you won’t even get an error message – all you will see is a blank page (another funny bug).

 

What about site collections?

If you have deleted a site collection, you no longer have the luxury of going to the Recycle Bin.  You have to switch over to PowerShell.  imageIf you fire up your PowerShell console window, you will find that SharePoint 2010 SP1 has introduced three (3) new commandlets designed to handle deleted site collections.

Unfortunately, the built-in PowerShell documentation for these commandlets are lacking.  You will need to pull down the information from MSDN for the new commandlets.

Command Link
Get-SPDeletedSite http://technet.microsoft.com/en-us/library/hh286316.aspx
Remove-SPDeletedSite http://technet.microsoft.com/en-us/library/hh286317.aspx
Restore-SPDeletedSite http://technet.microsoft.com/en-us/library/hh286319.aspx

 

If you provide no parameters to Get-SPDeletedSite, it will return the list of all deleted site collections.  This is particularly helpful if you happen to have more than 1 site collection with the same server-relative url.

image

Restore-SPDeletedSite will restore the site collection based on the parameters specified.  Finally, Remove-SPDeletedSite will purge the site collection from the recycle bin and thus allowing the site to be scheduled for absolute clean-up via the Gradual Site Delete timer job.

Points worth calling out...

There is a notable difference between these commandlets and others.  The SPDeletedSite commandlets only take server-relative urls.  This is pretty important to call out simply because this is a distinct change in behavior compared to other commands which normally take the full url.

6/29/11 update

NOTE: There is a difference between Remove-SPSite and Remove-SPWeb!

Remove-SPWeb has a new parameter -Recycle that allows an admin to route the deleted SPWeb over to the Recycle Bin.

Remove-SPSite did not change with SP1 and therefore all deletions bypass the Recycle Bin.

Finally, it’s also worth mentioning the SPDeletedSite commandlets only work if you have deleted the site collection via the browser ui or the object model.  If you are an admin and you have explicitly called Remove-SPSite, then your site collection is not available to the SPDeleteSite commandlets. 

-Maurice

June 24
Web Part Type IDs for SharePoint 2003, 2007 and 2010

Web Part Type IDs have always been something of a mystery for folks.  In short, web part type ids are unique guids that represent a web part type.  When the web part framework was built for SharePoint v2, our team decided that we needed a way to optimize the load sequence during a render cycle.  Out of the effort emerged a component known as web part type ids.  They are a way for SharePoint to clearly identify the types of each web part instance that is stored in the database while allowing the SafeMode parser to pre-cache the list of SafeControl types found in the web.config and hence run through the safe/not-safe validation quickly.

For developers, web part types ids are entities that we hardly ever ever need to manage.  SharePoint generates the web part types ids based on type and assembly information. However there are 2 phases where web part type ids start to bubble up in importance:

  1. Upgrade from SharePoint version to version
  2. Upgrade from Web Part version to version

For the latter, the general guidance that developers need to follow is that it is wiser to never use use binding redirect to upgrade the assembly version of a web part assembly. 

For the former, administrators will likely see errors in this space long before a developer is called into to help debug the problem. imageAdministrators will run into the problem as they run through the upgrade process from from v.old to v.next.  The problem will show up when explicitly running through upgrade or via the test-spcontentdabase commandlet. 

“Webpart class [some-guid] is referenced … but is not installed in the current farm”.  The error message is unfortunately not very informative; after all, web part type ids were never meant to be human consumable! Although, this is the primary exposure point, other pre-upgrade tasks and general web part operations may also generate message that include the web part type id. 

How are folks supposed to know what the guids represent?

Prior to SharePoint 2010, the answer was always buried somewhere.  If you are an old timer with SharePoint, you might remember a tool called the SharePoint Configuration Analyzer.  It’s a tool that I wrote somewhere in the beta time frame of v2 to help my former team gather information about the rendering characteristics of a SharePoint deployment as the beta rolled out.  The tool would surface the “normal” names alongside the web part type ids. 

If you are upgrading to SharePoint 2010, you still need some help.  Check out the pdf file WebPartTypeIDs I created this morning that outlines all the web part type ids from SharePoint 2003, SharePoint 2007, and SharePoint 2010.  It’s worth calling out a few caveats about the list.

Notes:

  1. The SharePoint 2003 list of web parts does not contain the SharePoint Portal Server controls, only the WSS controls.  I didn’t feel like installing SPS this morning.  :)
  2. The SharePoint 2003 list does include the optional add-on set of web parts that was called “the pickle” or more officially known as the “Office 2003 Add-in: Web Parts and Components”.  This add-on piece was most famously known for early charting controls and the WebCaptureWebPart.
  3. With the exception of 1 custom web part that was installed on all of test machines and The Pickle parts, this list represents only out of box SharePoint web parts.

How does SharePoint 2010 improve on the experience moving forward?

SharePoint now records the assembly name and type name whenever a web part is associated to a page.  imageThe type id is still a critical component of the page rendering process, but the additional information allows the error message to be friendlier and eliminates the need to use an external reference.  The message still contains the web part type id, but SharePoint 2010 now reports back the more human readable representation of the type in the form of both the type full name and assembly full name.  No more guessing as to what that funky guid represents.  Nice!

-Maurice

June 17
Announcing the Global Shape Remover for Excel Workbooks!

Are you tired of seeing the “Unsupported Features” warning whenever you view an Excel workbook on your SharePoint site?

Do you work with PowerPivot? If so, you are probably going to see this notification bar all the time. 

Excel Services will display a notification bar if it has detected that your workbook contains an object that it cannot render.  It take up space and really doesn’t give the user any tangible instructions on what is causing the problem.  Thus the workbook authors are left to randomly “clean up” their workbooks.

PowerPivot users will really like this feature… why?  PowerPivot automatically adds rectangles (i.e. shapes) around the slicers that control your pivot tables.  Depending on how you close the file, PowerPivot may not remove the rectangles.

What’s the generic fix? Remove the shapes manually before uploading to SharePoint.

Is that a smart way to do it?  Nope.  Simply put it’s too hard to go through all the worksheets in a workbook to remove the shapes before saving the file to a SharePoint site. Are you willing to do this every time you save your file, for every file?  We’re not!

The Global Shape Remover was designed to specifically handle the task of removing shapes from your workbooks!

Shape Remover ReportTurn on the feature and you’re done!  Any time you upload or edit a workbook, the Global Shape Remover will clean up your workbook for you. It works silently in the background! If the Global Shape Remover removes anything it generates a report that outlines what it removed and where from – you’ll have a worksheet that you can use to help find shapes if you decide that need to make a manual adjustment.

What’s even nicer about the Global Shape Remover? It’s sandbox-compatible!  That’s right – we’re talking about a handy tool that can be deployed without the need to involve a farm administrator!

If you a site collection administrator, all you need to do is upload the file to your Solution gallery and away you go.  How easy is that?

Give it a try!  Download a free trial version today and see how easy it is to get rid of that annoying warning dialog.

=====

Update October 10, 2011:

imageVersion 2 has just been rolled out and it now includes the capability to remove comments! 

In the report, each comment becomes a hyperlink to the original workbook location.  Now all your comments will appear in one place and you won’t loose them when viewing your workbook via Excel Services.

=====

May 26
Not all sandboxes are the same (aka working with Office 365)
I recently ran into a problem with an Excel project that I've been working on.  The code was specifically designed to work as a Sandbox solution.  (teaser: PowerPivot users will like this add-on!)
 
All was good.  Code was fully operational and churning along quite nicely in every test platform.  Then we decided to increase the test matrix to include Office 365.
 
Boom! 
 
We hit a hard wall - the solution wasn't even allowed to Activate.
 
It was clear - Office 365 was using a custom solution validator (see SPSolutionValidator) and my solution package was obviously causing validation to fail. That was highly disappointing.  The largest arena for my primary deployment target was unavailable.  It was time to throw on the investigative hat to figure where the problem specifically existed and how to fix it. 
 
After chatting with fellow MCMs, it's pretty clear that we've stumbled into a generic problem.  The Office 365 team has decided to place a set of restrictions on top of the normal sandbox limitations.  These restrictions are obviously something they have evaluated and determined to be worth implementing.
 
As a quick refresher, let's look at our deployment targets:
  1. Farm
    • Fully trusted
    • Partially trusted
  2. Sandbox
    • Out of box config
    • Custom restrictions via validators

As you can see now, solution validators are the game changers when it comes to working with sandbox solutions.  The real answer to our problem is understanding what rules are being enforced. 

With a list of known rules, you can obviously tailor your code to that environment.  Again, thanks to the MCMs, all of us now have a little more insight into what it takes to develop products for Office 365. 

Here's two resources that developers can reference:

It's worth calling out that the FxCop rules are brand spanking new thanks to Kimmo Forss, a fellow MCM and member of the Office 365 team.  The FxCop rules are still a work in progress as Kimmo and team work out a few other issues; however, with more info and tools, developers should have a better grasp of what is and is not allowed on Office 365.

Happy sandbox coding...

-Maurice

 

1 - 10Next
 
© Bluedog Limited 2004-2011
Microsoft Certified Master - SharePoint 2010 & 2007
Aptillon, Inc.
Microsoft Most Valuable Professional
Sign In