PowerShell for SharePoint in O365- Getting Started

Getting PowerShell for SharePoint Online up and running is relatively easy, however, you might have to download a couple of things. And if you are new to PowerShell, you could be wondering what can you actually do once you have the SharePoint Online Management Shell installed? So here is a beginners guide on how to get things started on your local machine.

1) Make sure that you have installed Windows PowerShell 3.0. If you do not have PowerShell 3.0, you will need to download the Windows Management Framework 3.0
2) You will need to install the SharePoint Online Management Shell, which can be downloaded from the Microsoft Download Center
3) Run PowerShell, Windows PowerShell, the new SharePoint Online Management Shell or the Windows Integrated Scripting Environment (ISE).
As seen in Figure 1, you can find your new Shell by searching for it. However which ever tool you decide to run, PowerShell, ISE or the Online Shell, you will need to run it as an Administrator. If you do not, you will receive an error, as seen in Figure 2.

Figure 1: Searching for the new Online Management Shell

Figure 2: You will need to Run As Administrator to avoid this error

If you take a close look at the error in Figure 2, notice the Import-Module cmdlet that is used to import the Microsoft.Online.SharePoint.PowerShell module. The cmdlet is using the "DisableNameChecking" parameter; this is because of the use of non-standard verbs. In Figure 3, you can see that the "Upgrade-SPOSite" would be the non-standard verb. You can view the imported cmdlets (verb-noun) by running:

Import-Module Microsoft.Online.SharePoint.Powershell -Verbose

Figure 3: Shows the non-standard verb

In my Windows 8 deployment, the module was automatically added, as seen in Figure 4.

Figure 4: Using ISE, you can verify installation by looking in the Modules drop-down.

One of the great features of running ISE is that users are able to see all the commands available to them. Figure 5 shows all of the commands (verb-noun) available for the Microsoft.Online.SharePoint.PowerShell module.

Figure 5: A list of all available cmdlets for the Online module

4) To be able to start using PowerShell online, you will need to connect to the admin site of your tenant.

Connect-SPOService -Url https://yourTenant-admin.sharepoint.com -Credential username@yourTenant.com

5) Once you are connected, take it for a test drive!

Get-SPOUser -Site https://yourTenant.sharepoint.com

In Figure 6, you can see the results and some interesting accounts used by O365 to help manage your site.


Figure 6: The returned results from the Get-SPOUser cmdlet.

Another benefit of using ISE over just the Management Shell,  is that the Commands tab within ISE will actually help me create my script by showing me the required and available parameters to fill out, as seen in Figure 7.

Figure 7: ISE will show you the parameters available and will insert the command into the Script Pane.

ISE also uses Intellisense, as seen in Figure 8.

Figure 8: Shows the Intellisense parameters available for the Connect-SPOService cmdlet.

If you are still a bit timid to start writing your own commands, an excellent reference for helping you to create PowerShell commands, is Bill Baer's online Windows PowerShell for SharePoint Command Builder. You can download the instruction guide or you can go straight to the command builder website and start creating.

Update: 08/09/2013: Cleaned up verbiage due to late night blog posting...

SharePoint, IIS, 503 errors and GPOs

One of the great things about my job is that I get to spend a bunch of time solving puzzles and fixing peoples' problems. I get to install SQL Server and SharePoint in existing, well established Federal environments. Incase you are unaware, in a segregated, secured environment, such as a Federal Agency, the left hand does not always talk to the right hand. The DNS team works apart from the Exchange Team, who works apart from the AD team, who is isolated from Network Ops team, who works independently from the firewall team, and so forth. Getting things accomplished in such an arena, can be challenging, and usually involves a vertical assent and decent approach, meaning I talk to my counterpart, they talk to their boss, who talks to, for example the AD Team's boss, and then my service accounts get created.
Scenario
I was working with a group out of Alaska and Seattle, WA, to get a SharePoint 2010 Enterprise environment with SQL Server 2008R2 up and running after the group finished a domain migration.
SQL Issues
As with any SharePoint Farm installation, I installed SQL first. Everything seemed to have installed correctly, however after reboot, the SQL Server instance and the SQL Agent instance would not fire up, as seen in Figure 1.

Figure 1: Showing the stopped SQL Server and SQL Agent accounts.
In the process of troubleshooting, I opened up the service instance for the SQL Server Service and verified that the password was correct. As seen in Figure 2, after entering the account password, I was greeted with a message.

Figure 2: Shows that after entering the account password, that the account has been granted the Log On As A Service right.
This pointed me in the right direction for the first problem in this environment. The service accounts that were created in Active Directory (AD) had a GPO rule that new accounts could not Log On As A Service. The new accounts were put into a new Security Group, ran a gpupdate /force (http://technet.microsoft.com/en-us/library/bb490983.aspx), rebooted the server and SQL was now able to be rebooted successfully and have the service instances come up running!
SharePoint Issues
Unfortunately, placing all the service accounts into the new Security Group did not stop all the issues within the Farm. SharePoint installed correctly, and Central Administration(CA) provisioned and started correctly, however, after rebooting the server, I would receive a 503 error when trying to get to CA. Typically you would receive a 503 error when the Application Pool Account for your site has been stopped. After manually starting the account and clicking on the CA link, I would get a 503 error and my Application Pool Accounts would be stopped again, as seen in Figure 3.

Figure 3: After browsing through the IIS sites, the Application Pools would stop.
Which brought up the following in the error logs:
Log Name:      System
Source:        Microsoft-Windows-WAS
Date:          6/28/2013 7:40:21 AM
Event ID:      5021
Task Category: None
Level:         Warning
Keywords:      Classic
User:          N/A
Computer:      
Description:
The identity of application pool SharePoint Central Administration v4 is invalid. The user name or password that is specified for the identity may be incorrect, or the user may not have batch logon rights. If the identity is not corrected, the application pool will be disabled when the application pool receives its first request.  If batch logon rights are causing the problem, the identity in the IIS configuration store must be changed after rights have been granted before Windows Process Activation Service (WAS) can retry the logon. If the identity remains invalid after the first request for the application pool is processed, the application pool will be disabled. The data field contains the error number.
Event Xml: (removed)
Pretty nice to actually get a message that means something for once. But which account needs the Batch Login Rights? This is Federal Environment, I just cannot ask for all of my accounts to be put into a different GPO. After reading this article on IIS 6 http://technet.microsoft.com/en-us/library/cc179801.aspx and this article that has the same issue,

http://talesfromitside.wordpress.com/2011/12/06/log-on-as-a-batch-job-permissions-removed/

, I was a little bit closer. Finally I read this KB article from Microsoft that really solved the problem:

http://support.microsoft.com/kb/981949 (see Figure 4)

Figure 4: Shows the default permissions and user rights for IIS 7.0, IIS 7.5, and IIS 8.0
So, after having the security policy modified to allow the IIS_IUSRS to Log On As A Batch Job, we ran a gpupdate /force, rebooted the server and the IIS Application Pool was able to stay up and running after trying to access Central Administration.

Update 12/29/2014
Recently, while working with a customer, I ran into interesting issue with my content service account (eg: spContent). I was able to provision my farm, create all of my service accounts, and create my Web Application and Site Collection. I was able to open Central Administration, however, when I tried to browse to the Site Collection URL, I would receive an error.

It was a bit odd that everything else worked, but the web page would not render.
Now, I was in an environment known for having issues with GPO and had a bunch of devices in the background doing all kinds of packet inspection, so I was a bit hesitant as to what could be the error.
Luckily I did find a very helpful blog post: SharePoint 2013: An exception occurred when trying to establish endpoint for context: Could not load file or assembly...
This article basically asks to make sure that Impersonate a client at authentication has local rights.

Summary

Make sure your Service Accounts can Log On As A Service and that your IIS_IUSRS are allowed to Log On As A Batch Job. You might need to have the Impersonate a client at authentication has local rights as well.
Make sure that you do an iisreset after you push your GPO updates (or reboot the server(s)).

Determine Max Users From Requests Per Second (RPS)

While doing some work for a client the other day, I needed to reverse engineer how many users can their farm handle based on the results from a Visual Studio Load Test. Determining the requests per second that the users will generate is easy once you plug in the required information based off of the end users' usage profiles.

Great!  According to the spreadsheet, their web server needs to be able to handle around 208 requests per second.  But if you run a load test on their farm, and you are given an Avg RPS, how many people can your farm handle based off the give utilization statistics?  This was a bit harder to figure out, but eventually I got there.  After plugging in the actual Avg RPS that the farm could handle while the hardware was still in it's "Green Zone" was 11.  When you put that into the spreadsheet, the number of Total Users that should access their environment should be less than 1,320.



You can download the spreadsheet from here: http://sdrv.ms/Xp7Sq0

Synchronizing Your Visio Stencils With SkyDrive

A colleague of mine, Ong (@OngYang), had upload the ITPro_PosterStencils.zip file to one of our project sites, and while I was downloading the zip over my MiFi, I started to thinking (insert joke here).  I thought to myself, that it sure would be nice to have those load into every one of my environments so that I do not have to keep reloading the stencils every time I open up Visio on every machine I use.  I mean, I use SkyDrive to synchronize up everything else, why not my Visio stencils as well.
If you do not have a SkyDrive account set up, you will need some other form of synchronization tool to synchronize your files to all of your computers (laptops, desktops, and yes, even tablets).  From here on out, I will assume that you are using SkyDrive.

1)  Go to your My Documents folder, find your My Stencils folder and move the My Stencils folder to your SkyDrive folder.  You might even already have a linked library folder called My Documents within your SkyDrive already (hopefully).

2)  Download the ITPro_PosterStencils.zip file from here and unzip it into a folder within your My Stencils folder.  You might even think about cleaning up the names of the stencils to make things more organized.

3)  Open up Visio.  Depending on the version of Visio that you are using, things will be a bit different for the next step.  This process is for using Visio 2013.

4)  Click on the General search link to get to your settings menu.

5)  From the general settings tab, find the Options tab.

6)  Within the Visio Options window, click on the Advanced tab.

7)  Scroll all the way to the bottom, and in the General settings section, click the File Locations...  button.

8)  Under the My Shapes heading, you can click the browse button and point to the new My Shapes folder within your SkyDrive.

9)  Click Ok and Ok and create a new drawing.
10)  Under more shapes, you should now be able expand out to see your new stencils.

By following these steps you should now be able to go to all of your Visio clients and point them at your synchronized My Shapes folder which will give you all of your Shapes when loaded.

Using PowerShell to Export Your .WSP Packages

During your upgrade from SharePoint 2010 to SharePoint Server 2013, we will need to make sure that we have all of the appropriate versions of our deployed WSP files.  If you have ever been a guest (consultant) on another farm, and your client has not been able to maintain a valid history of deployments, you could really run into a big upgrade problem.  Luckily for us, Shane Young, with help from Todd Klindt, wrote up a nice blog post on how to export and import your farm solutions using PowerShell called Using PowerShell to export all solutions from your SharePoint 2010 farm and other fun

I am only really worried about the extract portion of the blog, and have modified the script a bit to fit how I run my scripts.

#Get Backup Path
$bkdir = read-host("Enter Folder Location") # Get Backup Path!

# Set Backup Path if you want to hard code your path
#$bkdir = "\\serverName\Shared\Temp" (optional "C:\Temp")

# Verify folder exists
if ((test-path $bkdir) -eq $false ) # Verify folder else create it...
  {
     [IO.Directory]::CreateDirectory($bkdir) 
  }

# Add Snapin
Add-PSSnapin Microsoft.SharePoint.PowerShell -EA 0 

(Get-SPFarm).Solutions | ForEach-Object{$var = $bkdir + "\" + $_.Name; $_.SolutionFile.SaveAs($var)}

Thanks Shane for making life less stressful!

Microsoft SharePoint 2013 Upgrade Planning Template

I have modified the Microsoft Template, and you can find a copy of my upgrade template here: http://sdrv.ms/VqNni7

Getting Your SharePoint Farm Information

As we prepare our upgrade strategy for SharePoint Server 2013, one of the major requirements is documentation of our existing SharePoint 2010 environment.  This is also a good script to run when first getting your hands on an unknown farm.  This should save you a bunch of documentation time.
<Lecture>
In theory, you should be reviewing the output documentation of your environment on a weekly basis.  You could even go as far as importing the results into SQL, and tracking your growth and changes of your SharePoint Farm with SSRS.
</Lecture>
This script will create a folder, go through your farm, and create the appropriate .csv file as long as you have the appropriate permissions to run the script and you have permissions to access the data.

There are several blog posts on how to take all the .csv files and merge them into one .xlsx file.  Here are a couple that I found useful.  You will notice that they call the ComObject Excel.Application, so if you do not have Excel installed or the appropriate Office System Driver Connectivity Components installed, you will get an error.

This post by Jeff Hicks was very informational about how to get everything working:

http://www.petri.co.il/export-to-excel-with-powershell.htm

This page actually has a .ps1 for you to download, but in the Q&A section, imfrancisd has a script that works very nicely as well.

Update (01/10/2012)
Added -Limit All
Update (10/30/2014)
Added a check to see if Backup location ends with "\"
Added URL to Sandbox Solution location for output
Added site structure output for entire farm
Added Disposal of objects
Update (10/21/2015)
Moved Script to Gist
Fixed error in Get Features section thanks to Matthew Bramer (@ionline247)
Cleaned up Get Site Structure section to get ALL webs within each Site Collection by Web Application.

SPSRED: Upgrading your SharePoint and SQL Environments

I was fortunate enough to have scheduled the end of a client engagement to coincide with the start of SharePoint Saturday Redmond.  When I had submitted the topic, I was kind of hoping to not have it selected, as there was a lot of work that would need to be done.  That being said, here is my presentation:

Here is the farm layout:

 Here is the template that I used when creating SQL Cluster and HA Group:

Here is the output file from the Test-SPContentDatabase:

All files are located here:
https://skydrive.live.com/redir?resid=8E55AA8C038225F8!4265

Windows 8 and The Unsupported Cisco VPN Client

When you spend most of your time doing remote sessions into client environments using the Cisco VPN Client, I found it a bit troubling to find out that Cisco announced their VPN Client End of Life.  However, since I had the installation on a supported Windows 7 box, I was not too worried that Windows 8 would not be supported.  Fast forward to August 8, 2012: my laptop died, all I had was a Windows 8 tablet to use as my defunct work computer, and I was no longer able to get to my clients...
To resolve this issue:
1) Open up the Registry Editor:
\HKLM\SYSTEM\CurrentControlSet\Services\vpnva\DisplayName

2) Edit the DisplayName, and clean out all of the junk (including the semi-colon) before the adapter name, leaving just the display name.  For example:
"@oem10.inf,%vpnva_Desc%;Cisco AnyConnect VPN Virtual Miniport Adapter for Windows x64"
becomes:
"Cisco AnyConnect VPN Virtual Miniport Adapter for Windows x64"

Verify Your SQL Connection / Get Connection String

I have been very fortunate to have some fantastic mentors.  One of them is a gentleman by the name of Ken Ammann, who taught me this trick when you need to verify your SQL connection or get your connection string.

1) From your desktop, right-click and create a new text document.

2) Change the file extension from .txt to .udl

becomes:

3) Click Yes to verify the change of extension type.

4) Open the UDL file and set properties.

5) Test the connection.

6) Click OK.

7) Change file extension back to .txt

8) Take a look at your connection string.

It is a nice easy way to verify your SQL connections when you need to verify that you still have a SQL connection.  The real reason that I wrote this post is that I was spending too much time trying to recall the UDL extension name.