Delete Missing Features – dig deeper

I have been tasked with deleting missing features from existing farm. Now there are many articles on this. Here are a few very useful one:
Link 1
Link 2

 

Between the above 2 you will be able to tackle most of the features. Rarely though, you will need a bit extra. And I am thankful to Microsoft as I had to open a ticket to learn this. Sometimes, you will simply not see the feature through the command:
Get-SPFeature Your Feature Id
$db = Get-SPContentDatabase Your Content Database
$features = $db.QueryFeatures([guid]’Your Feature Id’)

Exception calling “QueryFeatures” with “1” argument(s): “Specified argument was out of the range of valid values.
Parameter name: Could not find Feature Your Feature Id.”
At line:3 char:1
+ $features = $db.QueryFeatures([guid]’ Your Feature Id’)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : ArgumentOutOfRangeException

This is when we have to a bit of digging into the content and config databases. You will need SQL Server Management Studio for this:

---

From Config Database:

Use Your Config Database
Go

Select * from SiteMap with (nolock) where databaseid = ‘Your Database Id where the feature is shown as missing’
Carefully check how many sites are in this database seeing the number of rows returned

---

From Content Database:

Use Your Content Database
Go

Select * from AllSites
Carefully check how many sites are in this database seeing the number of rows returned

---

Ideally both the above operations are supposed to return the same number of rows (sites actually). But you may find these numbers are somewhat off. Usually it is more in the content database.
When you see the above mismatch, here is how you try to resolve. Create a blank new website. Move the content database to that website. You can do it in Central Admin or can do it via PowerShell.
Once done, try removing the feature from the content database by running Phil Childss’ blog. Essentially by running:
$obj.Features.Remove($feature.DefinitionId, $true)
This time the feature will be found and the removal will be successful.
Once the feature is removed, bring the database back to the original web application.

Upgrade Required/ Upgrade Available – How to get rid of these messages from Central Admin

In 2017 I was mired with the pesky Upgrade Available/ Upgrade Required messages in DEV/ UAT and PRODUCTION.
Here is a summary what I learnt. Through my own and with some excellent Microsoft Engineers.
First thing you do is:
stsadm -o localupgradestatus
The above will be given in the Managed Servers in Farm screen if anything needs upgrade.
When it was run in PRODUCTION, I got the following summary:
[18] content database(s) encountered.
[0] content database(s) still need upgrade or cannot be upgraded.
[7605] site collection(s) are contained in the content databases.
[1] site collection(s) still need upgrade.
[41] other objects encountered, [1] of them still need upgrade or cannot be upgraded.

Here the line of interest is the underlined one. If you see the details, you will see messages like the following:

http://your site
Microsoft.SharePoint.SPSite
6
Needs Upgrade

At this stage, I tend to move the above site to it’s own dedicated database. And then detach and reattach. You can do it via the Central Admin. Or you can do it via:
Dismount-SPContentDatabase
Mount-SPContentDatabase

Here are some additional information:
Be fearless about clearing config cache. Do it manually. Do it here:
C:\programdata\microsoft\sharepoint\config
If you mistakenly delete the config cache.ini file, you will need to reinstall SP. Never do it, even in your worst nightmare.

Run the following from command line:
PSConfig.exe -cmd upgrade -inplace b2b -wait

If the above does not work, take it a step further and try the following:
PSConfig.exe -cmd upgrade -inplace b2b -wait -force

Sometimes you need to check and delete one time timer jobs that stay hanging.
To Check:
Get-SPTimerJob | where{$_.schedule.description -eq “One-time”}

Sometimes, you will see Upgrade Running forever. Try the following:
stsadm -o setproperty -pn command-line-upgrade-running -pv No

You may have to wait for good bit of time even after running the above for the Upgrade In Progress message to disappear.

Happy Upgrade Required removal.

Delete all items in a list using PowerShell

There are plenty of articles about it. However, I sometimes need ad-hoc, one line solutions. Here is the one works perfectly for me. However, remember, nothing will be in recycle bin:
$myList.Items | %{$myList.GetItemById($_.Id).Delete()}
where, $myList is the list. I usually get $myList by doing:

$myWeb = Get-SPWeb “Your Web”
$myList = $myWeb.Lists.TryGetList(“Your list name”)

Unexpected Crash For SharePoint Timer Service every few minutes

You may have seen this:
https://blogs.msdn.microsoft.com/shawa/2016/08/11/unexpected-crash-for-sharepoint-timer-service/

We have a SharePoint 2013 farm and it had the same exact message:
The SharePoint Timer Service service terminated with service-specific error %%-2147467259

It was being repeated every 2-3 minutes in one of the few servers we have in farm.

However, our resolution was different. We opened a ticket with Microsoft. With the assistance of the escalation engineer, we discovered the following line from the ULS log entries:
Could not determine SQL engine edition: System.TypeInitializationException: The type initializer for ‘System.Data.SqlClient.SqlConnection’ threw an exception. —> System.TypeInitializationException: The type initializer for ‘System.Data.SqlClient.SqlConnectionFactory’ threw an exception. —> System.TypeInitializationException: The type initializer for ‘System.Data.SqlClient.SqlPerformanceCounters’ threw an exception. —> System.Configuration.ConfigurationErrorsException: Configuration system failed to initialize —> System.Configuration.ConfigurationErrorsException: An error occurred while parsing EntityName. Line 113, position 96. (C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\BIN\OWSTIMER.EXE.config line 113) —> System.Xml.XmlException: An error occurred while parsing EntityName. Line 113, position 96.

The mistake I did is not to read the entire message. On the surface it seems like a SQL error. But on more serious scrutiny, I read this line:
An error occurred while parsing EntityName. Line 113, position 96. (C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\BIN\OWSTIMER.EXE.config line 113)

Here it points to OWSTIMER.EXE.config file. So we compared the OWSTIMER.EXE.config file of this server along with other servers and there were differences. Once the file was made consistent, the repetitive crash of the timer service went away.

Search Index – frequent degradation

We have SharePoint Server 2013 spread over 4 WFEs, 2 app servers. The search topology generally crawls using 3 servers and content processing also happens through 3 servers. Index replication is across couple of servers. We have multiple content sources. One content source is dedicated to crawling 1500+ team sites. The content size is about 1 TB. There are other content sources dedicated to a relatively smaller size like Intranet. All sites go through full crawls over the weekend.
For many months, we noticed that the team site full crawl causes degradation of index. Sometimes it degrades both replicas. We tried our best to troubleshoot. We opened tickets with Microsoft. Microsoft asked us many search reports.
We have page files on our servers. Our platform team determines that the page file does not have to be the recommended page file size and had it setup at a smaller value. Then one fine day, my supervisor wondered what if we make the page file size set up to the recommended value which is 1.5 times the physical memory on the servers. As soon as this was done, the problem went away magically. It’s been more than a month and the problem did not come back.

Excel Services Error – Part 2

Please refer to my last post. I was whistling and I asked my DBAs to check the same db permissions in PRODUCTION and they did. And I whistled some more in happiness of resolving a long standing problem. And guess what, in PRODUCTION my steps did not resolve the problem. And I kept on getting:

No problem. I recommended the use of ULS Logs to pinpoint the problem and I did. Nothing. No Excel Service related error reported. No problem at all. I turned verbose logging on and started looking for relevant error. Nothing. A lot of errors, but no Excel Services Related error. Days passed. Nothing. Checked event viewers as thoroughly as possible. Nothing.
I was wrapping up the troubleshooting on one long afternoon. And was about to give up as I did in the last few days. As a final resort, before shutting my laptop, I checked the Global Settings in Excel Services Application Settings. And I was simply looking through. I did not know what the default settings are. So I opened the same corresponding settings in another SharePoint environment and started comparing. I was startled to see that we changed some global settings in PRODUCTION and they are not consistent.
Once I made those settings consistent, low and behold, it started working. In the image below are before and default settings.

Bottom-line, troubleshooting means keep on looking. And then look for some more.

 

Excel Services Error

One fine day I get a case (ticket) from a user. We have seen these in the past. Excel Web Access web part giving generic error. User will say their page was working and now it does not work.
The error message will be very generic. Something like:

So I embark on troubleshooting. We have DEV, UAT (QA) and PRODUCTION environments. After some aimless wondering I discovered that our UAT environment has the same problem. I found this excellent post by Wictor Wilen:
Office Web Apps 2013: Excel Web App ran into a problem – not rendering Excel files
This told me not to give up and look in the right place. ULS logs. I generated the error in UAT and then merged up the ULS logs and looked under ULS Log viewer. And found the following error:
SQL Database ‘***’ on SQL Server instance ‘***’ not found. Additional error information from SQL Server is included below.  Cannot open database “Your Content Database” requested by the login. The login failed.  Login failed for user ‘Domain\yourServiceAccount’
Note that Wictor Wilen has found out a different error than mine. But the comb up the ULS log is what I needed.
After that, I asked my DBA group to add the necessary permission to the account to the content database and we are good to go.

Get Login-id for an Office 365 SharePoint site

I thought it was very easy to find. I go to any group that has the id and click on it and I will get something like this for the account:
i:0#.w|domain\userid
And I found it was way harder. I also tried the Get_SPOUser commandlet. It only gave me the userid part but not the token part.
Then a colleague asked me to try:
https://yourtenant.sharepoint.com/_api/web/currentuser
And in the LoginName field I found the answer.

 

How to wrap long names in PowerShell text output

Sort of a self note here.
Quite often we are asked to display all sites or all webs. I get frustrated to see the urls, which are often long truncated with ellipses at the end. The following formatting will help:

$myWebApplication | Get-SPSite -Limit All | ft -Wrap
$myWebApplication | Get-SPSite -Limit All | Format-Table -AutoSize

Where $myWebApplication is just a variable where I read the webapplication where I am trying to find the site collections.

Add web part to Edit Form of a list

I came to know that I can add content editor web part in SharePoint page and add javascript code to it. For a specific purpose, I needed to add a web part in a list’s Edit form.
I made sure that I set up Launch from Dialog to No as shown below:

And then opened the Edit form in IE 11 and tried to do Settings > Edit Page as shown below:

But nothing happened.
Then I tried the same thing in Chrome and it worked just fine.