Michael Phillips Blog

“netvsc” error in Hyper-V guest

We use Citrix Presentation Server for a number of applications, and lately we have had a significant increase in issues with one set of our Citrix servers.  We have 3 main sets of Citrix servers and the problems have only been happening on one set. 

One of the sets doesn’t have this error, but wouldn’t because they are physical servers.  They have been in production a long time, and we have plans to virtualize them. 

The second set doesn’t get the errors, but it is fewer servers and fewer users.

The third set:

• is virtual
• runs on 2008 R2 Hyper-V
• has more servers (6 as opposed to 4 or 5 for the other two)
• supports more users and more users per server (averages around 20 users per server during business hours)

Around November, we started upgrading our hosts from 2008 to 2008 R2.  The problems have been getting progressively worse peaking in the last 2 months.  Our last 2008 host was converted in March. 

After some event log review, we were able to correlate some of the issues to the following error in the event log:

Event Type:    Warning
Event Source:    netvsc
Event Category:    None
Event ID:    5
Date:        4/19/2010
Time:        3:49:53 PM
User:        N/A
Computer:    <ServerNameChangedToProtectTheGuilty>
Description:
The miniport ‘Microsoft Virtual Machine Bus Network Adapter #4′ hung.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 00 00 00 00 02 00 52 00   ……R.
0008: 00 00 00 00 05 00 00 80   …….€
0010: 00 00 00 00 00 00 00 00   ……..
0018: 00 00 00 00 00 00 00 00   ……..
0020: 00 00 00 00 00 00 00 00   ……..

and right behind that would be this message:

Event Type:    Information
Event Source:    netvsc
Event Category:    None
Event ID:    4
Date:        4/19/2010
Time:        3:49:53 PM
User:        N/A
Computer:    <ServerNameChangedToProtectTheGuilty>
Description:
The miniport ‘Microsoft Virtual Machine Bus Network Adapter #4′ reset.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 00 00 00 00 02 00 52 00   ……R.
0008: 00 00 00 00 04 00 00 40   …….@
0010: 00 00 00 00 00 00 00 00   ……..
0018: 00 00 00 00 00 00 00 00   ……..
0020: 00 00 00 00 00 00 00 00   ……..

After doing a bit of searching and getting a lot of nothing, and doing some on site troubleshooting without much luck, I finally broke down and called Microsoft.  I spent a day e-mailing back and forth with someone who was suggesting that I try all the things that I had already tried, so I contacted our TAM and had the case escalated. 

The technician then informed me that there was an internal hotfix that had not been fully tested yet, that related to my issue.  It seems that in 2008 R2 Hyper-V guests running Server 2003, the network adapter will hang and then reset under heavy load.  The hotfix has to be applied to the host and then the integration services on the guest have to be updated.  In my environment, when I updated, I had to remove the integration services from the guest before the updated NIC driver would install.  I reported this behavior to the technician I was working with, but he said that he couldn’t reproduce that particular problem and that he had no issues updating his test environment.

It is my understanding that the hotfix will be released under KB981836.  When you install this, it changes the integration services version from 6.1.7600.16385 to 6.1.7600.20683.  You can see this if you look at the driver version on the guest NIC.

Not recognized as a cmdlet…

I have been working on a simple little script to copy a file and then launch a program.  I am sure that there are a lot of ways to do it, but I decided to use PowerShell, and this is what I came up with:

$CheckForFile = "H:\custom.ini"
$FileToCopy = "c:\IT\custom.ini"
$CopyFileTo = "H:\"

$PathTest = Test-Path $CheckForFile
If ($PathTest -eq "false")
    {
    Copy-Item $FileToCopy $CopyFileTo
    }

#uses the Invoke-Item command to launch the application
Invoke-Item "C:\Program Files\executable to launch.exe"

This is for use in a Citrix/Terminal Server environment, so I want to be able to call this script like this: PowerShell copythenlaunch.ps1

When I tested that, I got this:

C:\IT>powershell copythenlaunch.ps1
The term ‘copythenlaunch.ps1′ is not recognized as a cmdlet, function, operable
program, or script file. Verify the term and try again.
At line:1 char:18
+ copythenlaunch.ps1 <<<<

I kept thinking there was some problem with the install of PowerShell (I am running this particular script on a Windows 2003 Server) or that I had some illegal character in the name (it had a number in it originally) or some other simple problem.  Finally I did a search and came across this little bit of conversation:

re: Power and Pith

I just started with PowerShell.

Wanted to run some test scripts from you download.

When I tpye in Beep.ps1 I get "The term ‘Beep.1′ is not recognized….."

What Am I doing wrong?

Friday, December 29, 2006 3:17 PM by MikeL

# re: Power and Pith

> When I tpye in Beep.ps1 I get "The term ‘Beep.1′ is not recognized….."

> What Am I doing wrong?

You are relying upon a traditional bad shell behaviour that has been a security nightmere for decades.

In PowerShell, you have to be explicit if you want to run a command in the current directory.  Type ".\beep.ps1"

Jeffrey Snover [MSFT]

Windows PowerShell/MMC Architect

Visit the Windows PowerShell Team blog at:    http://blogs.msdn.com/PowerShell

Visit the Windows PowerShell ScriptCenter at:  http://www.microsoft.com/technet/scriptcenter/hubs/msh.mspx

Friday, December 29, 2006 5:19 PM by PowerShellTeam

# re: Power and Pith

Thank You for supplying the ".\*" information.  I have been racking my brain for almost two days wondering what I was doing wrong.  And to think it was as simple as using the PROPER .\yourscripthere.ps1 format.

Thank you very very much

Ditto on the thanks…

Windows PowerShell Blog : Power and Pith

TechNet wiki…

So a TechNet wiki is coming, as stated by Keith Combs here:

three big things we focus on for all TechNet scenarios are Content, Discoverability, and Participation. We really want to invite participation from everyone and what better way to combine that with discovery and content than to use Wiki technology?

Later this year TechNet and the Server & Cloud Division will partner to launch the new TechNet Wiki.
…
We believe a public wiki for technical content on TechNet has the potential to be a big step forward in all three areas:

• For content, while Microsoft teams write some great content, we cannot possibly author all the content customers need to be successful, and a wiki that offers content from the community will increase the breadth and depth of technical content available for IT Pros.
• For participation, the wiki is a great new way for any IT pro to create and share technical knowledge, engage with other technical people and with Microsoft, and build an identity and reputation within the TechNet community.

I used the word "potential" above because Microsoft cannot succeed with the TechNet Wiki on its own – success ultimately depends on the direct engagement, support, and ongoing feedback from the IT community.

I am not sure how I feel about this.  I like Wikipedia, and OFTEN reference it for information on non-work related topics.  We use a SharePoint wiki for a lot of our documentation where I work, but I don’t find it to be as useful as I would like, because for some reason, I don’t like the interface.  One of the big issues I have with it, is the inability to insert pictures directly in the article during the creation of the article.  You have to upload the picture (such as a screenshot) separately and then link to it.  That doesn’t seem very efficient to me.

With this blog, I use Windows Live Writer.  It allows me to take a snip like this:

and paste it right in the text.  When I publish, it is auto-magically uploaded and linked to my post.  I think the other issue that I have is that I am afraid I would take it personally if someone felt the need to correct something  I posted to a wiki.

I guess I am going to have to work on that.  Both for the wiki at work and (maybe) for the TechNet wiki.  There are those occasions when I actually have useful knowledge to share…

Influencers Blog

So the System Center guys have provided a place for people who work with System Center products to see a conglomeration of posts from various professionals who have registered to Blog about System Center products.  How fun…

Blog Posts by System Center Influencers

Get the feed.

Below are the most recent posts from several of the members of the System Center Influencers Program. Note that Microsoft does not review the content or endorse it in any way; we present this content in a feed form for your information and convenience. (In the event that the feed refuses to render due to the flakiness of the third-party feed service, simply use the feed embedded in the RSS icon above.)

Nexus SC: The System Center Team Blog : Blog Posts by System Center Influencers

An online pack already exists…

Situation:  I have a 32bit file server running Windows Server 2003.  I have a new 2008 R2 (x64) file server running on Hyper-V.  Did I mention that the R2 server is on Clustered hosts?

This is all simple.  Just drop the LUNs from the file server, give them to the cluster and assign them as pass through disk to the guest OS.  Simple…

Except, I kept getting the following errors when I try to do the import:

An online pack already exists.

The operation failed, because an online pack object already exists.

The provider encountered an error while merging two packs.

I mean really…  what does all that mean.  I couldn’t find anything pertinent in my searches, I tried working in Diskpart, and got some of the same errors.

After lots of searches, I couldn’t find anything to help.  So I opened a ticket with Microsoft.  I got a very helpful person who told me that it basically sounds like a driver issue, but he couldn’t say for certain.  He couldn’t find much on it either.

So on a whim, I moved my test LUN from my production file server to another 2003 server.  That worked just fine, so I then moved it from there to the 2008 R2 machine that I had been working with.  That worked just fine.  Weird…

So I checked the driver for the production file server:

and compared that to the other 2003 server:

As you can tell, the culprit seems to be an outdated driver on the current file server.  Seems I don’t keep things as up to date as I should.

Error installing Integration Services on Hyper-V VM

I was trying to change from a “Legacy Network Adapter” to a “Network Adapter” on one of my Hyper-V VMs.   I added the “Network Adapter” and removed the “Legacy Network Adapter”, and started the machine up.  When the machine came up, it wouldn’t connect to the network.  Having seen this before I knew exactly how to fix it.  Run the Integration Services install again.  I did that, and got:

An error has occurred: One of the update processes returned error code  61658

So, being the smart guy that I am, I rebooted and tried again.  Same result.  The hits I got on google suggested that I was still running an CTP or Beta, but I am running the RTM version (2008 Hyper-V, but not R2 on this one). 

I was logging into this machine with a Domain account, but it wasn’t on the network.  We use some restrictions on the server desktops and redirect application setting and such, so I thought that might be related. 

Here is what worked:

I added back the Legacy Network Adapter (leaving the non-Legacy adapter as well)

I installed integration services again (and it worked just fine)

I removed the Legacy Network Adapter

Now the machine is working just as expected.

Error installing DPM 2010 Beta

I was installing the DPM 2010 Beta (finally) and had an issue trying to get the SQL 2008 to install.  Finally figured out that I had the install files stored too deeply in a network share.  I figure this out by running the SQL install directly and when it when to check prereq’s it had an error on one section and when you click for more info this is what you get:

Rule "Long path names to files on SQL Server installation media" failed.

SQL Server installation media on a network share or in a custom folder can cause installation failure if the total length of the path exceeds 260 characters. To correct this issue, utilize Net Use functionality or shorten the path name to the SQL Server setup.exe file.

So, I moved it to a shorter path and it installed just fine.

Data Protection Manager 2010

So I am a bit late realizing this, but the Beta for DPM 2010 is available now on the Connect site.  I haven’t read anything on it yet, so mainly I am just posting this to make myself look into it.

https://connect.microsoft.com/Downloads/DownloadDetails.aspx?SiteID=840&DownloadID=22070

Remote Server Administration Tools for Windows 7

As a follow up to my earlier post about the Remote server Administration Tools for the Windows 7 Release Candidate (RC), I thought I would post a link to the RTM version.  It took a bit to find it.  A search on Microsoft’s site wasn’t helpful to me.  I finally found a reference on Ben’s Blog here: http://blogs.msdn.com/virtual_pc_guy/archive/2009/08/12/rsat-for-windows-7-rtm-now-available.aspx

Which sent me here:

Remote Server Administration Tools for Windows 7

Brief Description

Remote Server Administration Tools for Windows® 7 enables IT administrators to manage roles and features that are installed on computers that are running Windows Server® 2008 R2, Windows Server® 2008, or Windows Server® 2003, from a remote computer that is running Windows 7.

Download details: Remote Server Administration Tools for Windows 7

Temporary Registry Profiles – Finally an answer

We have been having issues at work for a while where a user gets “cannot load the locally stored profile” and a temporary profile is created.  It gets to be quite frustrating for the users because each time they log in they lose any settings that they have made.  Apparently, we arent’ the only company that has been having this issue.  The complete article has a very interesting discussion on the Windows Logical Prefetcher and what the cause of the problem was, but the gist of what you need to fix the problem is update your Citrix client or use the work arounds mentioned below:

Now that the problem was understood, Microsoft and Citrix brainstormed on workarounds customers could apply while Citrix worked on an update to the ICA Client that would prevent the sharing violation. One workaround was to disable application prefetching and another was to write a logoff script that deletes the Ssonsvr.exe prefetch files. Citrix published the workarounds in this Citrix Knowledge Base article and Microsoft in this Microsoft Knowledge Base article. The update to the ICA Client, which was made available a few days later, changed the network provider DLL to 10 seconds after Ssonsvr.exe launches before returning control to Mpnotify.exe. Because Winlogon waits for Mpnotify to exit before logging on a user, the Logical Prefetcher won’t associate Winlogon’s accesses of the user’s hive with Ssonsvr.exe’s startup.

Mark’s Blog : The Case of the Temporary Registry Profiles