Mrdenny

Recently I’ve been working on a system where I had to create a test script that reliably recreated a database each time it was executed. I’ve done this many times before and thought I knew how but I was wrong. It turns out that reliably dropping a database in a script that you run from SQL Server Management Studio (SSMS) is harder than it looks. (It’s also harder than it should be).

In Books Online (BOL), the article for DROP DATABASE says :

“You cannot drop a database currently being used. This means open for reading or writing by any user. To remove users from the database, use ALTER DATABASE to set the database to SINGLE_USER.”

Based on this, I had been using the following style of script to drop and recreate a database:

USE master;
GO

IF EXISTS(SELECT 1 FROM sys.databases WHERE name = N'Blah')
BEGIN
    ALTER DATABASE Blah SET SINGLE_USER WITH ROLLBACK IMMEDIATE;
    DROP DATABASE Blah;
END;
GO

CREATE DATABASE Blah ON

...

However, what I’ve found is that when I execute this from SSMS, it doesn’t always work. Fairly randomly (and hard to reproduce), it fails with an error message telling me that the DROP failed because the database was in use. But isn’t that the whole purpose of setting it to single user?

The problem seems to be that although the database gets set to single user, another connection could be made to it before the DROP DATABASE statement occurs. Setting the database to SINGLE_USER isn’t enough as the current database context for the script is the master database, not the database in question.

A number of users and fellow MVPs have told me that they’ve experienced the same issue. What we suspect is causing this is the Intellisense system within SSMS connecting to the database to check the syntax of objects later in the same script. The problem only really seems to happen in SSMS but it could happen elsewhere.

A great suggestion from Rob Farley was to set the database OFFLINE instead of SINGLE_USER. While that would stop the other connection, the problem is that when you DROP a database that is OFFLINE, the files for the database don’t get deleted. You’d then need to have some additional script (nasty) to delete the files as well after the database was dropped.

What is really needed to get around this race condition is:

DROP DATABASE WITH ROLLBACK IMMEDIATE;

If you agree, I’ve created a Connect item here that you can vote on:

https://connect.microsoft.com/SQLServer/feedback/details/813014/cannot-reliably-drop-a-database-in-t-sql-script

This is the fifth, the final part of the fraud detection whitepaper. You can find the first part, the second part, the third part, and the fourth part in my previous blog posts about this topic.

The Results

In my original fraud detection whitepaper I wrote for SolidQ, I was advised by my friends to include some concrete and simple numbers to calculate the return on investment (ROI) in a language that managers can understand. with some customers, we really managed to get very impressive numbers. However, I am not repeating this calculation here. However, this is my personal blog. Therefore, I am writing my personal opinion here.

I am kind of bored with this constant requests to show simple numbers that managers can understand. Personally, I don’t think that managers are that stupid that they would not understand anything beyond primary school mathematics. And even if some of them are that dumb, I don’t care, as they can’t become my customers. They would never be able to understand the value of such an advanced technique like data mining.

I am pretty sure that the vast majority of managers can calculate approximate ROI by themselves, and also better than I can do. They definitely know their business better than I can do, and already know how much money they are losing because of frauds and how many frauds they are already preventing or catching early. In addition, I am pretty sure that most of the managers do understand the value of learning, and appreciate building of the learning infrastructure.

Therefore, in short, I am leaving to you, to the reader, to evaluate what can you expect from implementing a fraud detection continuous learning cycle. And thank you for understanding my point!

Conclusion

Fraud detection is a very popular, albeit very complex, data mining task. I have developed my own approach to fraud detection. The most significant element of this approach is the continuous learning cycle.

Although Microsoft SQL Server is not the most popular tool for data mining, I am using it. The SQL Server suite gives us all of the tools we need, and because all of the tools come from a single suite, they work perfectly together, thus substantially lowering the time needed to bring a project from the initial meeting to a production-ready deployment.

Another advantage of my approach is the mentoring with the knowledge transfer. It is not my intention to get permanent consulting contracts; I want to progress together with my customers. Once we finish the project, or sometimes even as soon as we finish the POC project, the customer can begin using and continue improving the fraud detection system constantly with the help of the continuous learning infrastructure.

Finally, due to Microsoft’s licensing policies, the customers that already possess Microsoft SQL Server Standard Edition or higher and Microsoft Excel, do not need to purchase any additional licenses.

Read more of this story at Slashdot.

Read more of this story at Slashdot.

Read more of this story at Slashdot.

Peter was cautiously optimistic before the busiest part of open class enrollment season at his employer - Northeastern Institute of WTF. Peter and his team worked hard to implement a new user-friendly website for students to enroll in classes. Now, the brain-dead students who couldn’t figure out how to register should call the service-desk less. He even contracted a third-party datacenter with “state-of-the-art load balancing hardware” to handle the web traffic stress on the system. This would be the simplest enrollment season ever!

Three days before the enrollment deadline, all the procrastinating students (i.e. all the students) swarmed the website at once. Instead of being greeted with the shiny new enrollment site, they found the dreaded “Page Unavailable” screen - if they even got any response at all. This in turn loaded the IT support office’s phone lines with calls of “OMG I waited until the last minute to register for classes and now I can’t!!! I NEED Survey of Underwater Basket-Weaving or I won’t graduate!”

Peter and his staff of three got overwhelmed, so they flipped the switch on the phone system, activating the “We’re sorry, all lines are busy right now” prompt to stop the calls. With this chance to breathe and figure out the problem, Peter dug in to the network logs. There where myriad connection errors. The student traffic got to the remote load balancer but didn’t get a response.

Peter got on the horn with the datacenter support staff and explained the problem. A man named Roger gave him the runaround from the datacenter, denying it was a problem on their end. Through Peter’s persistence, he finally got them to agree to look at the load balancing device and servers it routed to. “Now we play the waiting game…” Peter said to his guys as they started up a foosball match while the department’s phones were still ignoring everyone’s calls.

After what could have been an entire World Cup’s worth of foosball, Roger finally called back. “I’ve got bad news and bad news. Which would you like to hear first?” Roger snarkily said. “Um… bad news?” Peter said before gulping.

“Our load balancer is using the default settings, which is simple round-robin,” Roger explained. “Requests alternate between the two servers, regardless of how loaded those servers are.”

“Ok, so you can just change that setting and everything will be ok, right?” Peter replied with hope.

“Well now, that’s the other bit of bad news. We also have a randomized server backup system in which one of the servers in our datacenter is chosen each week to handle backing up all the other servers. Unfortunately, the server this week is one of yours. We aren’t able to stop the backup without messing things up, and it has 18 terabytes more to get through. So, it’s going to be about 48 hours before your server farm can handle anything other than mild stress,” Roger stated ominously.

When Peter hung up, he felt like he’d been gut-punched. He instructed his subordinates to draft an e-mail to all students and faculty to explain the open enrollment system wouldn’t be usable before the end of the week. Peter took on the task of crafting an e-mail to the dean of Northeastern Institute of WTF with about 25 apologies and a request that the class enrollment period be extended a few days. Peter then put his head down on his desk and considered firing himself for not vetting the third-party datacenter before signing the contract. This suddenly became the most difficult enrollment season ever.

[Advertisement] BuildMaster 4.0 is here! Check out the brand-new UI and see how you can deploy directly from TeamCity (and other CI) to your own servers, the cloud, and more.

I meant to be all organised this year and have something posted on NYE about how rad 2013 was and all the cool things that I am planning to do in 2014. But I didn’t. And as I’m woefully incompetent when it comes to these style of posts, I’ll keep it short.

Here’s some of my personal highlights from 2013:

• Got selected to the inaugural EMC Elect group of 75;
• Got selected as a VMware vExpert for the first time;
• Made some decent progress with the DIY Heatmaps script (that’s really all due to Mat though);
• Passed some more accreditations;
• Went to Europe for a nice long holiday and discovered that not speaking French in the last 20 years hasn’t helped me stay “courant”; and
• Kept this blog going in spite of my occasional desire to kill it off.

In 2014, some things will change and some will stay the same:

• I’m looking to contribute more to the various on-line communities;
• I’m starting a new contract with a local SI so you’ll be seeing a few more posts “from the front”; and
• I’m getting a bit more focused on rebuilding my home lab.

Site-wise, I find that, if nothing else, this blog has been a useful form of catharsis and I imagine it will remain that way in the foreseeable future. Stats-wise, I received a 10% increase in hits over 2012. This is due in no small part to the exposure afforded me by the nice people running EMC Elect and VMware vExpert programmes. My goal, however, is not to keep increasing stats but to publish articles that are useful. Based on the last few years, that may not be as simple as it seems.

Enough rambling. Enjoy 2014, and above all, stay healthy and don’t worry too much about all this nerdy crap we seem to get so hung up on from time to time.

Transition is always a topic of fascination. It’s one close to me as I continue to process leaving EMC for Infinio just a month ago.

While all experiences are somewhat unique, I found my timing wasn’t. 

Two friends of mine in the technical world, one being Ian Massingham, left their previous roles and started new jobs on the exact same days as I did.

Coincidence? Completely. Opportunity? Absolutely.

I interviewed Ian on a number of questions I was asking myself at the time. His responses are insightful for others thinking of where their career could bring them next. I want to share them as a bit of a hat tip to others as they go throughout their career.

Note: You’ll see he responds in the Queen’s English, which I did not correct out of respect for his personal verbiage.

Q: Can you give a brief review of what job you recently left and where you are now?

I was previously CTO for EMC’s Service Provider Program in the Europe, Middle East and Africa and part of EMC’s Global Alliances organisation. This role included building the pre-sales Systems Engineering function supporting EMC’s service provider partners and also an evangelism and marketing component focused on recruiting new SP partners and on enabling existing partners to develop and sell ‘EMC Powered’ Cloud and IT Service offerings.

I’m now at Amazon Web Services as a Technical Evangelist.

Q: What’s the first word that comes to mind when you think of your first day and why?

Intense.

That probably encapsulates it pretty well. There’s so much to learn in any new role, but this feels like a super-amplified version of that.

Q: Do you have a memorable “overheard” moment or just a moment that stands out?

Definitely receiving some great on-boarding advice from some of my new team members.

<< Note: All of which had to be redacted due to Ian not yet having credentials to speak on internal events >>

Q: If you had to choose just one, what’s the one way in which our community helped you in this transition?

This is a tough one to answer, because there are quite a few aspects that I could focus on.

I think the key one is the extremely positive reaction and support that I have received since I announced that I would be moving on from EMC. Switching jobs can be a stressful experience, especially when you have the responsibility of supporting a young family, so it was great to be able to count on some good friends within the community to support me in my decision.

Transition is guaranteed. As I continue to learn from my personal experience, I find reflecting on my former colleague’s movements to be just as informative as my own.

Ian has a great blog on his experiences thus far and I’m sure he’ll add more as they occur in 2014.

What questions do you have about career movements? Ian and I are happy to talk more.

[One more post recovered from SQLCAT.com archives.]

Author: Sanjay Mishra, Tim Wieman

Technical Reviewers: Mike Weiner, Shep Sheppard, Prem Mehra, Chuck Heinzelman, James Podgorski, Silvano Coriani, Madhan Arumugam Ramakrishnan, Sunil Agarwal, Guy Bowerman

Windows Azure SQL Database and SQL Server in Windows Azure Virtual Machine are two different data storage options and are best suited for slightly different application scenarios. Refer to the published articles on MSDN for a comparison and how to choose between these options.

There are certain differences between the two storage options. One important different is the isolation level of the database. Depending upon application design, difference in isolation level can impact the application concurrency, and ultimately, application performance and throughput.

It is not a common practice to port databases from Windows Azure SQL Database to SQL Server in a Windows Azure VM environment. However, for some specific unique reasons, not covered here, one of our customers performed such a migration and the only change made was to change the connection string.

During the load test after the porting, we noticed that the performance of the application on SQL Server in Windows Azure VM did not match up to the performance of the application on Windows Azure SQL Database. Upon further investigation, we found much higher volume of lock waits under SQL Server in Windows Azure VM.

The key difference that impacted the application behavior is the isolation level of the two databases. On Windows Azure SQL Database, the isolation level settings for READ COMMITTED SNAPSHOT and SNAPSHOT ISOLATION are turned on:

select name, snapshot_isolation_state, is_read_committed_snapshot_on from sys.databases

On a SQL Server database (running on Windows Azure Virtual Machine, or on-premises), the isolation level settings for READ COMMITTED SNAPSHOT and SNAPSHOT ISOLATION are turned off, by default:

select name, snapshot_isolation_state, is_read_committed_snapshot_on from sys.databases

The customer application was designed and developed on Windows Azure SQL Database and it did make use of READ COMMITTED SNAPSHOT ISOLATION (RCSI).

The performance comparison of the two data storage options, under default settings:

As you can see, the application throughput for SQL Server in Windows Azure VM was almost one-third of the throughput with the database on Windows Azure SQL Database.  One of the important goals of the load testing was to ensure application response time under 2 seconds. For SQL Server in Windows Azure VM, under default settings, the application response time was more than the 2 second threshold the customer had set. Analyzing waitstats perfmon counters (counter “Lock Waits” under object => “SQL Server:Wait Statistics”, instance => “Average wait time (ms)”), we observed about 1 sec average wait time for locks.

With RCSI turned on, the average lock wait time came down to a few milliseconds, application throughput significantly improved, and the application response time significantly improved.

Important Note:  When turning on RCSI, be aware that SQL Server maintains the version store in the TEMPDB database, and you may observe significant activity in TEMPDB.  Therefore, appropriate capacity and throughput must be planned for TEMPDB.
For example, for this specific scenario, after enabling RCSI, the increased throughput demands of TEMPDB necessitated scaling out TEMPDB data files to multiple data disks.  Following is the TEMPDB disk I/O throughput comparison before and after RCSI:

Further Reading:

• Windows Azure SQL Database and SQL Server -- Performance and Scalability Compared and Contrasted
• Isolation Levels in the Database Engine

This post is a continuation of the SQL Server 2014 Cardinality Estimator enhancements exploration series:

• A first look at the query_optimizer_estimate_cardinality XE event
• “CSelCalcCombineFilters_ExponentialBackoff” Calculator
• “CSelCalcCombineFilters_ExponentialBackoff” Calculator– Part II
• The CSelCalcAscendingKeyFilter Calculator
• Cardinality Estimation Model Version
• Comparing Root-Level Skews in the new Cardinality Estimator
• Using Legacy Methods to Lessen SQL Server 2014 Cardinality Estimator Skews
• More on Exponential Backoff
• Non-Join Cross-Table Predicate Correlation Changes

This post, like the previous nine posts on this subject, uses SQL Server 2014 CTP2 as a reference point.  There may be changes by SQL Server 2014 RTM, and if so, I’ll write a post about applicable changes.

Now in terms of troubleshooting the new Cardinality Estimator, what I’m specifically referring to is the introduction of cardinality estimate skews that negatively impact query performance compared to the pre-SQL Server 2014 cardinality estimator functionality.   Ideally performance regressions should be rare, but when they happen, what are our troubleshooting options?

To frame this discussion, let’s first discuss what may or may not warrant action…

No Action (Necessarily) Needed

• The estimates are identical to old CE functionality and the query plan is unchanged
• The estimates are skewed compared to the old CE functionality, but the query plan “shape” is identical (and you see no side-effects from the skews, such as sort or hash spills and query runtime degradation)
• The estimates are skewed compared to the old CE functionality, the plan is different, but performance is equal or improved – or even more stable
• The estimates are skewed compared to the old CE functionality, the plan is different, and performance is somewhat impacted but not enough to justify changes (totally depends on your SLAs & workload performance requirements of course)

Action Potentially Needed

• The estimates are skewed, the plan shape is unchanged, but the estimates lead to performance issues such as spills (due to under-estimates) or concurrency issues (due to over-estimates) for memory-intensive operators
• The estimates are skewed, the plan is changed, and the plan quality leads to performance degradation (a variety of query optimizer plan choices which may lead to issues)

So what troubleshooting methods and options are available to us?

Troubleshooting Options

• Do nothing (yeah, I know, but this can be a decision you ultimately make, looking at risk/effort/reward)
• Revert to the pre-SQL Server 2014 CE version (for example, via database compatibility level change)
• Apply legacy troubleshooting methods, which may fix other issues directly or indirectly related to the skew and thus help close the gap (framing these legacy methods as questions below)
  • Are the statistics old and need updating?
  • Should the statistics sampling be changed?
  • Are multi-column stats needed to help establish a correlation where one currently isn’t seen by the query optimizer?
  • Parameter sniffing troubleshooting needed? (a much larger topic, but indulge me on including this as a method)
  • Is your table variable usage contributing to the skew?
  • Is your multi-statement table-valued function or scalar user-defined function contributing to the skew?
  • Any data-type conversions occurring for predicates (join or filter)?
  • Are you comparing column values from the same table?
  • Is your column reference being buried by a function or embedded in a complex expression?
  • Are hints being used and if so, is their usage appropriate?

The new CE story will unfold as customers start upgrading to SQL Server 2014 and I’ll be curious to see which regression patterns are most common.

Open Questions

• Regarding the new query_optimizer_estimate_cardinality XE event… Will it be a practical source of information for most SQL Server users in cardinality estimator skew regression scenarios – or will it be something reserved for edge-cases and advanced Microsoft customer support scenarios?  I suspect this XE event will have limited applicability, but I’m reserving judgment for now.
• Will SQL Server 2014 RTM introduce finer-grained methods for reverting to the pre-2014 cardinality estimator?
• How will the new CE behave with newer functionality?  For example, Hekaton and clustered columnstore indexes.
• Will this be it for CE changes for the next few versions?  There is plenty left on the CE-improvement wish list, so I hope not.

The post Troubleshooting the new Cardinality Estimator appeared first on Joe Sack.

Read more of this story at Slashdot.

Read more of this story at Slashdot.

With the year drawing to a close, it's a great opportunity to review our best articles from 2013. Here's one written by snoofle that was originally published back on January 31st.

Telecommunications manufacturing is a cut-throat business. Features, functionality and hardware need to be added and continuously improved at a frenetic pace in order to stay one step ahead of the competition. Engineers must constantly increase their skills to leverage the latest advances in technology to design and build the best product possible at the lowest cost. Slip up just a little, and it can be a death knell for your company.

To save costs, Dog and Bone Corporation (DNB) had eased up on their latest-and-greatest push. Budgets shrank. Progress on adding new features slowed. Hardware upgrades failed to happen. Meetings to justify every little thing began to become the main task in everyone's day. Competitors leapfrogged. Interest waned. Unfortunately, this caused their flagship private telephone exchange (DBX) product to start to stagnate and made some of the engineers begin to feel that their skills were getting to be a little out of date. Engineering-eyes began to wander.

In the white hot heat of competition, DNB needed something to revive sales of their flagship DBX. If it happened to persuade some engineers to stay because there was actually something interesting to work on, then so much the better.

Duncan E. was on the team that helped develop one of the new features: an interface to permit an external computer to not only receive notification of telephone activity within the DNB exchange but to control features and behaviour of the exchange itself! Perhaps not particularly astonishing by standards of today, but in 1990 this was cutting-edge stuff. The latest tools were brought in and the team was trained in a suitably advanced formal methodology, so as to be best equipped to produce an excellent product.

Lo and behold, over many long months the team laboured: analysing the requirements, capturing state transition behaviour, and designing the gateway that would provide the magic window into the telephone exchange. When the magic window was switched on, it worked remarkably well - a strong indicator that formal methods were a valuable tool for DNB's future. For once, management had done it correctly; they gave the engineers the tools, training and blessing to do it right, and it worked. The engineers were also quite pleased with their accomplishments.

Then management reverted to their usual meddling and it began. Delete this. Add that. Change this. Then one of them mentioned that they had read an article about hackers and software piracy.

The DNB management team was worried that nasty software copying thieves would come and take the program from them, stealing all the hard work and effort that the development team had poured into this paragon of software. They wanted their investment in the program to be protected. Period.

And so it was decreed that a dongle would be added. Without this dongle the software would not run. Without this dongle the software would be without worth. Without this dongle, there would be no way to control the DBX. Software would run amok. Skynet would arise. Empires would crumble. And horror of horrors, phone calls might even be rerouted so that only robo-telemarketing calls could be received.

It was Duncan who implemented the dongle code. He took great pains to insert a macro to do the dongle checking rather than using a function call (so as to make it harder for software pirates to hack the dongle-present check).

It was also Duncan who realized that the dongle was an effort in futility, although he left before he told anyone. After all, the software wouldn't run without a DBX to talk to, making the telephone exchange itself the biggest dongle in the world.

[Advertisement] BuildMaster 4.0 is here! Check out the brand-new UI and see how you can deploy directly from TeamCity (and other CI) to your own servers, the cloud, and more.

Dear Santa,

My friend and debate foe victim Thomas LaRock ( blog | @sqlrockstar) sent you some last minute gift advice for organizations who need your help.  That got me thinking, so I made a list, too.  I checked it three times because I’m thorough like that. I’ve based this list on observations and data I’ve collected over the last year.  Much like you do.  Except I didn’t use that creepy shelf elf guy.  

Just in case you didn’t collect enough data, or it got lost in some hard drive failure:

For Twitter: A way to restore those 20k Tweets of mine they lost a few years ago.  Oh, wait…probably better that they aren’t part of the firehose any more.

For Data Modeling tool vendors: A copy of my multi-volume set of enhancement requests for supporting new database and datastore types. It’s nearly 2014; it’s time we data architects were able to do our job regardless of the target technology.

For the NoSQL Community: A love of data so strong that they can tolerate the mere mention of relational database solutions when appropriate.

For Microsoft: A brilliant new name for Windows Azure SQL Database. I can’t keep saying that in my presentations.  Even WASD is difficult. 

For Oracle, IBM, and Sybase users: A community as active and helpful on social media as those in the SQL community.

For United Airlines: A set of laminated copies for each employee of your Star Alliance agreement to remind you that 125k flyers do indeed get perks on your airline. First World Problems, I know.

For Star Alliance airlines, including Air Canada: A "Wifi in the Sky for Dummies" book. Tell them to get cracking.

For NASA: That other half a penny.

For Justin Bieber: US citizenship and a ranch in SoCal.  And a plane ticket.  No monkeys.

For Rob Ford:  NULL.  Not even coal.

For my readers and their customers: A year in which their data is much loved, in the right place, at the right time, with the right granularity and the right integrity.  Or World Peace.  Probably easier.

Love,

(Kitty is my Starbucks name.  I figure Santa knows that already.)

I originally wrote this in December 2011. It’s been one of my most popular posts, so reposting it has become a Christmas tradition. Enjoy!

The users were nestled all snug in their beds,
Blind to the pressure on the poor admins’ heads
A rep from the vendor, dressed to impress
In case of a problem, he’d ease the distress

From the server room there arose such a clatter,
We all sprang from our chairs to see what was the matter
First one to the door, was there in a flash
I swiped my key card, expecting a crash

Lights, rows and rows, continued to blink
Assuring me that we a good link
Proceeding, across the room I ran
The sound it seemed was an exploding fan

Without a good fan the temperature would rise
Repair would be needed, and to my surprise
The rep was in action
Calling for help his immediate reaction

More rapid than eagles his supporters they came,
And he whistled, and shouted, and called them by name;
“Now, Sheldon! now, Leonard! now, Howard and Raj!
Hurry, bring in those parts from the garage!

To the back of the rack, over there by the wall
Now dash away! dash away! dash away all!”
True experts they were, their hands they did fly
Removing the cover on the very first try

In a matter of seconds, the fan was all new
The techs were so proud, and the rep was too
And then, in a twinkling, I heard a low beep
A text from my boss, asking “How deep?”

I quickly responded “Don’t worry, we’re good”
As the rep proclaimed from where he stood
“It’s time, we should get started”
As his helpers departed

A flurry of buttons he proceeded to click
An obvious pro, this guy was quick
His eyes — how they twinkled! his dimples how merry!
His cheeks were like roses, his nose like a cherry!

His droll little mouth was drawn up like a bow,
And the goatee on his chin was as white as the snow;
A dribble of Red Bull ran down his chin,
I had no doubt this guy liked to win

He had a broad face and a little round belly,
That shook, when he laughed like a bowlful of jelly.
He was chubby and plump, a right jolly old elf,
And I laughed when I saw him, in spite of myself;

A squint of his eyes and a twist of his head,
Gave reason to think there was something to dread;
He spoke not a word, but continued to work,
Then suddenly logged off and turned with a jerk,

And laying his finger aside of his nose,
And giving a nod, from the chair he rose;
Grabbing his jacket, he cried “That’s it, we’re done!
I love doing upgrades, they’re so much fun!”

Then I heard him exclaim, as he drove out of sight,
“I’ll send you a bill, but not tonight!”

Read more of this story at Slashdot.

Read more of this story at Slashdot.

Before upgrading to vSphere 5.5, I ensured that my host profile was up-to-date and on all nodes. After upgrading, it is important to continue update the host profiles immediately after upgrading. However, the normal way to update profiles, using the vSphere .NET Client, may not work. When I tried it, no matter which host or profile, and whether new or old, the profile would not update. I received a range error. A quick search found nothing but did hint at a few things. Many days later, I restarted the .NET client and noticed once more the message at the top that says:

“The traditional vSphere Client will continue to operate, supporting the same feature set as vSphere 5.0, but not exposing any of the new features in vSphere 5.5.”

Well, I thought, perhaps that is it. Host Profiles has many new features associated with vSphere 5.5, so we are now forced to use the vSphere Web Client for these features but must use the .NET client for VUM, SRM, and a few other solutions. Not a major problem, so I updated the host profile within the web client from one node. It failed with a check error but not much else.

That node is rather special, as it has a bit more hardware in it and is not what I consider my normal node. To that end, I went to a more basic node and updated the profile from it. It was successful. Then it was time to edit the profile to remove any of the storage issues that tend to pop up from time to time. What I did can be found in a previous post in my vSphere Upgrade Saga. The next step after editing the profile is to check compliance against it (see Figure 1).

Figure 1: Host Profiles in vSphere Web Client (Click to Expand) Note the circle used to check compliance

Within the vSphere Web Client, I feel there could be serious scale issues with a large number of hosts within a host profile. I am not sure; what do others think?

But that comment aside, when I used Host Profiles on vSphere 5.5, it was the first time that all nodes were compliant after I applied the profile. While it was still necessary to remediate the profile on all nodes, it was also required to once more disable some of the storage elements within the host profile. One day, I expect that will be fixed. Remediation through the web client was fairly straightforward. All you need do is to put the node in maintenance mode and then remediate directly from the host profile page of the web client.

Figure 2, Remediate and scan (click to expand). Note the first circle to remediate and the second to rescan

With the web client, it is important to refresh the screen after you remediate, as sometimes the host compliance is just not updated yet.

Unfortunately, there are still some configuration issues that show up within VMware vCenter (Figure 3), as well as within vCops as a Health issue and in HP Insight Control as an alert related to those items in Figure 4.

Figure 3: vCenter Configuration Issues

While it is possible to tell the tools to ignore the vCops and HP Insight Control issues, it is best to fix the problems by using the network interfaces to provide management network redundancy as well as to provide a spare NIC (Figure 4).

Figure 4: Root Cause of unlinked errors in vCops

To fix the management network redundancy issue, we can make use of vmnic6 on each host and attach it to a second administrative network with its own IP. Or we can ignore the error by following the instructions within KB1004700. Or, using the latest firmware from HP for Flex-10, we can prevent virtual NICs from showing up on the hosts. However, I prefer to assign these physical NICs so that I can remove one more cause for error from my systems and have less I need to remember to do when adding hosts. The last NIC we can attach to an alternative network for later use. Since this is all within my blade enclosures, the work takes place within the HP Virtual Connect Manager. Those steps are:

Step 1: Create two new Ethernet networks within HP Virtual Connect Manager

Being the creative soul I am, I created one Ethernet network that is named ADMIN2 and another named ALTERNATE. These names are really unimportant; you can use anything. I also tied ADMIN2 to an external network on the back of the blade enclosure (which took a bit of hands-on cabling and cable management to accomplish).

Step 2: Place a target node in maintenance mode, then power off

By placing the node in maintenance mode, you can safely migrate off all virtual machines to other nodes. The node needs to be powered off to manipulate the server profiles within HP Virtual Connect Manager. This is where SVMotion and vMotion come into play.

Step 3: Modify the server profile

Once the node is fully powered off, it becomes possible to edit the server profile within HP Virtual Connect Manager. Simply add the networks that have just been connected to the profile. This is accomplished by pressing the big “+” sign, then selecting the network names from the drop-down list.

Step 4: Apply the server profile

If the blade is fully powered down and everything is correct, then saving the profile will succeed. If not, then fix the other issues and re-apply the profile.

Step 5: Power on the blade

Power on the blade using the power switch, HP BladeSystem Onboard Administrator, or HP ILO power on the blade. I tend to want to do all my system maintenance from the comfort of my own desk.

Step 6: Meet the network redundancy requirements

Add a second vSwitch with a vmkernel management port on it. Give this second management port an IP address. Different subnets are just fine, as it is used by HA for heartbeat, etc. While I have network redundancy in hardware, this adds network redundancy in software and alleviates several problems.

Now repeat all these steps for all nodes, and you are just about good to go.

Step 7: Update host profile and verify compliance

The last step is to once more update the host profile from the source host and verify compliance using the vSphere Web Client. What I found odd once I applied the host profiles was that I had to go back into the security settings of each host and stop the ESXi Shell. I would have expected Host Profiles to take care of this for me, just like it did for the start and stop controls for the ESXi Shell.

Host Profiles has vastly improved, but it still has some problems here and there. It is extremely helpful when you want to keep systems the same from a networking, firewall settings, and advanced configuration perspective, but there are still issues with local storage and disabling services.

These changes allowed my systems to be compliant and removed all the warning icons from my hosts, some serious errors in other management tools, and a Health issue from within VMware vCenter Operations. This gives me a good baseline from which to see errors related to vSphere 5.5 and anything new with my underlying hardware.

I don’t do a lot of baking. My kitchen is mostly the place where I blend my breakfast and enable my caffeine addiction. But my family has a tradition of making dozens and dozens of cookies every holiday season. Sugar cookies, No Bake Cookies, Snickerdoodles…the list just goes on and on.

As I was looking in my pantry for ingredients this year, I started thinking about how the process of producing cookies was a lot like data architectures. I may have been drinking. I’m pretty sure of it, actually. A lot. I mean I’m a lot sure I might have been drinking. A lot.

This week I bring to you a short series about Christmas Cookies and data.

Sugar Cookies

Yum! Probably the most common version of Christmas cookie is the decorated, cut out sugar cookies. Recipe books, blogs and food network shows make them look so easy. They contain just a few simple ingredients (butter, sugar, flour, salt, vanilla, eggs) that form the basis of almost all other higher forms of cookies.

What makes these special is what you do with that dough. The most exciting versions have you to roll out the dough, cut it out with cute cookie cutters, bake, cool, then decorate them. It’s just cutters and icing, right?

The Big Lie

I’m here to tell you that it’s all a lie. First, unless you have a lot of practice, the dough never rolls out cleanly because a whole lot of things have to go right first. Then you cut them out and they fall apart or tear. You’ll end up burning the first few batches until you know how your oven heats and how your baking pans work.  Maybe you need at Silpat liner. Or parchment paper.  Or an actual baker.

But no amount of equipment prepares you for the disaster of decorating them. They NEVER come out like the pictures. Those cookies on blogs and in recipe books are probably made by specialist magical cookie elves who spent their 10,000 hours learning to make cookies from Betty Crocker herself.  With Photoshop. I’m pretty sure every decorated cookie recipe is shopped worse than a Ralph Lauren model.

There are all kinds of warnings in the recipes: let the cookies cool on a rack. But who has time for that? Be agile and decorate them while the cookies are still in the cooling sprint. Oh. Crap. What the heck happened? If you haven’t spent a lot of time doing some test and training baking, your first set of cookies are going to be an embarrassment.

Silver Balls, Silver Balls…

And did you know that those little silver and gold balls that are the key part of the most beautiful cookies ARE NOT SUPPOSED TO BE EATEN? It says so right there on the label, “To be used as a decoration, not as a confection”. I bet you didn’t even RTFML. You’ve been unintentionally poisoning your kids and grandpa for decades. Or maybe intentionally. I won’t ask.

Lessons Learned

What does this teach us about data?

1. Recipes make everything look easy. A lot of people see the recipe books and assume that making these cookies is very easy. And yet it’s difficult to get them right. The dough needs to be the right temperature and have the right ratio of ingredients to make the dough the right consistency. This requires not just a recipe, but a lot of practice.  It also requires good technique, the right tools and the right environmental factors.

   The same thing applies to data architecture. Sure, one can watch a 45 minute presentation on what all those boxes and lines are, but until they have applied the principles then lived with the results of their practice designs, they won’t really understand why one cannot just use melted butter or leave out the baking soda because it’s easier. It takes a lot of experience to be a good architect. Just like it takes a lot of experience to make beautiful decorated cookies.
   

2. Demos of data modeling and design tools make everything look a lot easier than they are in real life. Part of this is because demos take time to give and they have to deal with the easy case. Sure you can migrate a database from Oracle to SQL Server by running a wizard. But you might not like the database or the data that comes out the other end. In fact, I can guarantee it you won’t. Migrating from one infrastructure to another always requires analysis, design, and implementation expertise. Decisions, even. Tools are never a substitute for design.
   
3. If you are an amateur, you’re going to make a lot of mistakes. Heck, even professionals will make mistakes. But amateurs are going to make more.  It’s how it works.  You make mistakes, learn from them, get better. You’re going to burn a lot of data, and therefore users and ultimately customers.  You can read all the recipes in the world and watch all the episodes of Iron Chef, but living with the results of your design decisions is what helps you learn. It’s okay to make a lot of mistakes if you are learning in a class. Or are working on a development project iteration.

   Production, though, is like learning to cook your first meal for Christmas dinner for a close family of 20-30 people. It doesn’t scale well and you’ll just end up disappointing everyone in a big way. Heck, you might even kill some people with your bad design.  You might have some letters after your name, but until you get to the professional level, don’t call yourself a chef.  Well, you can, but your customers aren’t going to trust you after the second batch.
   

4. You need to read and learn. Warning labels are a good start. The great think about most data principles is that they haven’t changed a lot. The technologies have, but not the foundations.  If you don’t read and learn, you won’t be in a position to deal with change that is coming whether you want it or not.
   
5. Some ingredients for data actually don’t really help the data. Comma delimited data in a column is fast. It allows people to go around the whole data governance process. Stuffing internal-only customer data in to AddressLineFour is fine, right? Until someone prints that on the envelope and mails it to the customer. Sure, these cute workarounds are shiny and happy. You need to be able to see when people are proposing the equivalent of shiny silver balls. They are pretty, but not for use in real life. You can quote me on that.
   

There are probably a lot more lessons to be learned from Sugar Cookies, but I just wanted to cover the basics. Just like the ingredients for Sugar Cookies.

Protection of data has always been a priority for customers, and disaster recovery allows the protection of data with better restore times and lower data loss at the time of failover. However, as with all protection strategies, additional storage is a cost that needs to be incurred. With storage usage growing exponentially, a strategy is needed to help enterprises control their spend on storage hardware. This is where data deduplication comes in. Deduplication itself has been around for years, but in this blog post we will talk about how users of Hyper-V Replica (HVR) can benefit from it. This blog post has been written collaboratively with the Windows Server Data Deduplication team.

Deduplication considerations

To begin with, it is important to acknowledge the workloads that are suitable for deduplication using Windows Server 2012 R2. There is an excellent TechNet article that covers this aspect and would be applicable in the case of Hyper-V Replica as well. It is important to remember that deduplication of running virtual machines is only officially supported starting with Windows Server 2012 R2 for Virtual Desktop Infrastructure (VDI) workloads with VHDs running on a remote file server. Generic VM (non-VDI) workloads may run on a deduplication enabled volume but the performance is not guaranteed. Windows Server 2012 deduplication is only supported for cold data (files not open).

Why use deduplication with Hyper-V Replica?

One of the most common deployment scenarios of VDI involves a golden image that is read-only. VDI virtual machines are built using diff-disks that have this golden image as the parent. The setup would look roughly like this:

This deployment saves a significant amount of storage space. However, when Hyper-V Replica is used to replicate these VMs, each diff-disk chain is treated as a single unit and is replicated. So on the replica site there will be 3 copies of the golden image as a part of the replication.

Data deduplication becomes a great way to reclaim that space used.

Deployment options

Data deduplication is applicable at a volume level, and the volume can be made available with either SMB 3.0, CSV FS, or NTFS. The deployments (at either the Primary or Replica site) would broadly look like these:

1. SMB 3.0

2. CSVFS

3. NTFS

Ensure that the VHD files that need to be deduplicated are placed in the right volume – and this can be done using authorization entries. Using HVR in conjunction with Windows Server Data Deduplication will require some additional planning to take into consideration possible performance impacts to HVR when running on a volume enabled for deduplication.

Deduplication on the Primary site

Enabling data deduplication on the primary site volumes will not have an impact on HVR. No additional configurations or changes need to be done to use Hyper-V Replica with deduplicated data volumes.

Deduplication on the Replica site

WITHOUT ADDITIONAL RECOVERY POINTS

Enabling data deduplication on the replica site volumes will not have an impact on HVR. No additional configurations or changes need to be done to use Hyper-V Replica with deduplicated data volumes.

WITH ADDITIONAL RECOVERY POINTS

Hyper-V Replica allows the user to have additional recovery points for replicated virtual machines that allows the user to go back in time during a failover. Creating the recovery points involves reading the existing data from the VHD before the log files are applied. When the Replica VM is stored on a deduplication-enabled volume, reading the VHD is slower and this impacts the time taken by the overall process. The apply time on a deduplication-enabled VHD can be between 5X and 7X more than without deduplication. When the time taken to apply the log exceeds the replication frequency then there will be a log file pileup on the replica server. Over a period of time this can lead to the health of the VM degrading. The other side effect is that the VM state will always be “Modifying” and in this state other Hyper-V operations and backup will not be possible.

There are two mitigation steps suggested:

1. Defragment the deduplication-enabled volume on a regular basis. This should be done at least once every 3 days, and preferably once a day.
2. Increase the frequency of deduplication optimization. For instance, set the deduplication policy to optimize data older than 1 day instead of the default 3 days. Increasing the deduplication frequency will allow the deduplication service on the recovery server to keep up better with the changes made by HVR. This can be configured via the deduplication settings in Server Manager –>File and Storage Services –> Volume –> Configure Data Deduplication, or via PowerShell:

Set-DedupVolume  -MinimumFileAgeDays 1

Other resources:

http://blogs.technet.com/b/filecab/archive/2013/07/31/extending-data-deduplication-to-new-workloads-in-windows-server-2012-r2.aspx

http://blogs.technet.com/b/filecab/archive/2013/07/31/deploying-data-deduplication-for-vdi-storage-in-windows-server-2012-r2.aspx

Read more of this story at Slashdot.

In my IT career, one of the things I have found that sets me apart is my ability to write. As IT pros, we write a lot. Whether we're talking email or documentation, senior level IT workers are always writing. However, not all of us graduated college with a degree in English. I certainly didn't. The good news is you don't have to have an English degree to write well. Writing well consists of understanding the rules, observing good writing, and practicing.

One of the biggest areas that folks struggle is with respect to grammar. A grammatical mistake can obliterate an otherwise well-written message. Here are some of the big offenders:

CopyBlogger: 15 Grammar Goofs That Make You Look Silly

Do you struggle with any of these rules? The best way to overcome the problematic rule is to study the rule and then practice using it. For instance, the rule on there, their, and they're is one a lot of people get wrong. There refers to a place, their is possessive, and they're is a contraction of they are. If I was having an issue with this rule, I'd try to write 20 sentences of each example. After a few days of practice, I'd have the rule down. If you've never tried this exercise, it works on the concept of deliberate practice.

Master these 15 rules as a start to writing better. Writing, like most other skills, is a skill you can always improve on. Professional writers are always writing. They are always trying to improve. Even though writing isn't our primary skill, it is still an important one for any IT worker. Invest time and effort in your writing and you greatly benefit your career.

Hello Friends,

In a recent consulting engagement, I was researching on the edition of SQL Server 2012 to be recommended to the client for upgrading their PowerPivot model on Excel 2013(Personal BI) to SSAS tabular (Enterprise BI). Considering that the size requirements were not very demanding I thought Standard edition SSAS tabular would be the right fit, but to my surprise found out that SQL Server 2012 Standard edition does not support SSAS tabular at all. This is very surprising as this kind of information is hard to find and I was expecting the option to be available though the feature set may vary (like for the SSAS Multidimensional)

Thanks

Veera

Haven’t had time to blog much lately. Finding out strange and interesting things about SQL Server 2014 to present at some seminars and conferences next year. “Shameless self-promotion” posts for those events will be forthcoming. ;-) A few days ago, an experiment with tracking compiled stored procedures turned up some “interesting” results. Thought I’d write about that.

You normally get information about query execution at a query level with sys.dm_exec_query_stats, and info at a stored procedure level sys.dm_exec_procedure_stats. However, with in-memory, OLTP the point is to make things run as quickly as possible and, as such statistics (such as worker_time and elasped_time) don’t get collected. By default *nothing* appears in these DMVs about compiled sprocs. Not even a count of the number of time it’s been executed.

However, Books Online mentions using sys.sp_xtp_control_query_exec_stats and sys.sp_xtp_control_proc_exec_stats to enable collection. Although sys.sp_xtp_control_proc_exec_stats controls a global setting, sys.sp_xtp_control_query_exec_stats controls collecting query information on a per-proc basis. That is, you can turn in on/off for individual compiled stored procedures.

I needed a test bed to try this out. Since there is a nice, new in-memory-OLTP sample on Codeplex that layers on top of the AdventureWorks2012 database, and being the lazy person I am, I decided to use this one. Shouldn’t have been so lazy, perhaps. I downloaded the sample, downloaded AdventureWorks2012 and found interesting behaviors right off. After attaching the data (mdf) file with ObjectExplorer, I found the the database had no owner (owner is blank). That ruined the sample setup, so I changed the database owner to ‘sa’. Then the sample setup ran. Next, I issued the statement:

select * from sys.sql_modules where uses_native_compilation = 1;

This produced output naming two stored procedures (good) and four triggers (not so good, triggers can’t be natively compiled). One for the Connect site.

Strangely the native procedures were named Sales.usp_InsertSalesOrder_inmem and Sales.usp_UpdateSalesOrderShipInfo_native. While I wondered why they both weren’t suffixed by _inmem or _native, I found additional stored procedures in the demo that ended in “_inmem” and WEREN’T compiled stored procedures, they just referenced in-memory tables. One for me to remember when looking at the results.

Started by clearing the procedure cache (DBCC FREEPROCCACHE, standard disclaimer not to do this on production) and inserting 10000 rows and updating additional rows with collection off (the default). Took 7 seconds. After this experiment there were no rows for those procedures/queries in the two DMVs. Good. Now turn global settings on for both types of collection, free proccache, and run again. Executes in 4 seconds. Something needed some “warmup” because I would have expected “with collection” to take longer. OK let’s look at the DMVs.

For query_stats, there are 5 queries in the two compiled sprocs. I get sql_handle (for the query), start/end offset, creation/last execution time, and the worker_time and elasped_time buckets. Nothing else. I don’t know what possessed me to think there’s be something in read/write counts, but these were zero, because of the way memory-optimized tables are stored. They’re not in stored in pages. Procedure_stats had the same sparse infomation, which I was used to by then.

In case you were wondering what the query plan looked like, there wasn’t one exposed. Plan_handle was zeros for those queries. In SQL Server 2014 CTP2 you still can’t get an actual query plan (with actual counts), but you can get an estimated plan. The estimated plans are really sparse as far as information goes, compared to what you’re used to.

One final surprise was that, I thought I’d run multiple iterations of the test, using DBCC FREEPROCCACHE in between to zero the numbers. Not only does turning off data collection not zero the numbers, but running DBCC FREEPROCCACHE doesn’t get rid of the rows, either. These rows remain in the DMV until I drop and re-create the natively compiled proc. Or restart SQL Server, of course. That’s something I wasn’t expecting, but logic-ing it out a bit, perhaps this information wasn’t retrieved from the plan cache. Sure enough, looking at sys.dm_exec_cached_plans yielded no rows for these plans, because they’re not traditional plans.

That’s enough fun for now, hopefully. More, on related topics, perhaps, coming…sometime.

@bobbeauch

The post Tracking execution stats of SQL Server 2014 natively-compiled sprocs appeared first on Bob Beauchemin.

In my past upgrade sagas, I had upgraded vCenter and then fixed a few niggling problems (or attempted to). Now it is time to actually upgrade the vSphere servers. I previously staged the vSphere 5.5 ISO into the VMware Upgrade Manager (VUM). Since all nodes need to be rebooted to do the upgrades, it is also a good time to update firmware. Otherwise, the upgrade is pretty straightforward. Hopefully, it will fix my remaining issues.

vSphere 5.5 Upgrade

To upgrade vSphere to 5.5, I did the following for each node:

Step 1: Enter Maintenance Mode (and evacuate all VMs, not just the running ones)

This step keeps my VMs running with zero downtime, which is a major feature of using virtualization for me. Without the ability to vMotion and migrate workloads, I would have to shut down mission-critical components. If your VMs are on local drives, you will have to migrate them by hand to other drives. This is where shared storage comes into play.

Now, if you are like me, you will have some failed vMotions due to mounted CD-ROMs. This is another reason I make entering maintenance mode my first step. I remove CD-ROMs and migrate the running VMs by hand. Then, any shut-down VMs will automatically migrate.

There are also times when your VMs are linked to local storage, such as a virtual storage appliance. To finish placing he node in maintenance mode these types of VMs need to be migrated off any local storage devices using SVMotion. Then, the virtual storage appliances can be shut down, but they will not migrate, as the VSA VMs are attached to local resources. This is also where redundant virtual storage appliances are useful as it alleviates the need to migrate non-VSA VMs.

Step 2: Upgrade Firmware as Needed

This is an optional step, but since we are rebooting nodes, it is a good time to install any firmware updates. For HP hardware, I tried using HP SUM remotely but decided to do an offline upgrade (which also uses HP SUM) instead, as it gives me a bit more control over what gets updated. You should go with what you are used to. However, for larger environments, some form of scripting is the way to go.

Step 3: Using VUM, Remediate the Upgrade

Since we staged the install ISO into VUM, we now need simply to remediate the upgrade. You need to select upgrades for this to work properly. Then, you can select the image associated with the upgrade.

Step 4: Using VUM Remediate Extensions

Now that the upgrade is done, it is time to update your extensions, which could include hardware-specific items (in my case, HP offline bundles), Nexus 1000V upgrades, and additional vendor extensions to vSphere, among others. Step 4 and Step 5 are interchangeable and depend on how you like to do upgrades.

Step 5: Using VUM Remediate Patches

After the extensions are upgraded, remediate any patches. VUM will not do upgrades, patches, and extension updates all at the same time; you need to do them separately. Step 5 and Step 4 are interchangeable and depend on how you like to do upgrades.

Step 6: Reboot the Node

I always reboot a node once more, just to be sure. After all those upgrades, perhaps something did not start properly. In some cases, the CIM server may not have started (as was the case after adding the HP SMX Providers).

Step 7: Exit Maintenance Mode and Power On Any Local VMs

Now, we are ready to place everything back in the cluster for use by virtual machines. I generally power on any necessary VMs, such as VSAs.

Repeat these steps for each node in the cluster; a rolling upgrade is the way to go. For those clusters using auto-deploy, there are other steps to go through.

Now that my cluster is at 5.5, it is time to consider building a new Host Profile to ensure hosts stay compliant. This caused a few issues for me, as I kept getting a range error, but before I touch on host profiles, we need to upgrade some of our management tools. Hardware management is just as important as VMware vCenter upgrades. While we did this once before, that was before the nodes were upgraded to vSphere 5.5, and this time I want full integration for better overall management.

HP Insight Control Upgrade

I also had to ensure that all my management tools worked with vSphere 5.5, so the next task was to ensure that everything worked with HP Insight Control since we not only upgraded vSphere but also upgraded the firmware on each blade. I upgraded HP Insight Control to the newest available version, v7.3. The steps I took were:

Step 0: Make a Snapshot of the HP Insight Control Virtual Machine

This is one of the more important steps, because it allows you to recover from major upgrade disasters faster than if you just use a backup.

Step 1: Upgrade HP Insight Control to v7.3

This is a simple upgrade that I’ve done many times. However, I first had to clean up enough space to allow the upgrade to happen. I had 8.5GBs of items that I could delete from my Microsoft Windows 2008 R2 system. That is quite a few upgrade packages and other items.

Step 2: Subscribe to WBEM Events

Within HP Insight Control, you can use Options -> Events -> Subscribe to WBEM Events to subscribe to these events. Doing so is required for HP Insight Control for vCenter as well as for the HP Matrix Operating Environment. However, this could fail for several reasons:

1. The HP extensions for HP management are not installed: To fix, simply install the HP management VIBs. I install four, which you can get directly from HP.
2. The node needs to be rebooted, as the HP SMX Provider for ESXi’s CIM was not started: To fix, simply exit maintenance mode and reboot the node. The reboot is required, as sometimes just restarting CIM services via the vSphere Clients does not work.
3. There are problems with the HP Extension VIBs, usually indicated by a Java.lang.error when trying to subscribe: To fix, remove all HP VIBs, reboot, and then reinstall the VIBs using your favorite method. To remove, I used ESXCLI, but I used VUM to reinstall them after a reboot.
4. There are partial WBEM subscriptions within the ESXi system: To fix, call HP Support and ask for the Python script DeleteInstanceWBEMSub.py. To use this script, you are required to have pywbem installed on the system. This is available as a package with the CentOS and RHEL6 releases of Linux. A Windows version of Python and pywbem are also available. HP Support may claim this is not available for your version of HP SIM, HP Insight Control, or HP Matrix Operating Environment. However, it is, it works, and it fixes the problems. After you delete the WBEM subscriptions, you can re-identify the host within HP Insight Control and then resubscribe.

Unfortunately, there are times when a re-identify does not work. When that happens, after using the Python script, I remove the host (and its ILO) from HP Insight Control, rediscover it, and re-delete the partial subscriptions; then I am able to subscribe to WBEM events. There could also be a combination of errors, like problems with the HP Extension VIBs, followed by a need to clean up partial subscriptions.

This may seem a bit nitpicky, but if your management tools do not work properly, specifically those for your hardware, you end up with many alarms and issues that could easily be solved by looking at, for example, HP Insight Control for vCenter. There are also HP Insight Controls for KVM, Hyper-V, and what looks to be a full series of hypervisor and cloud infrastructures. In addition, if WBEM is not properly subscribed to in HP Insight Control, it could affect how data is seen inside VMware vCenter as it also uses WBEM to communicate to get hardware health information.

There you have it: I am now updated to vSphere 5.5. Next, I will try to fix my Host Profiles issues.