Sunday, November 9, 2014

How VMware snapshots affect Veeam backup efficiency?

Recently I was given task to take over one VMware environment which has Veeam backup in use. When going through the environment, I found out that some backup jobs take way more time that what I have usually seen and started investigating this issue.

First thing that I found was, that there are lot of snapshots in that environment, also in those servers that are backed up with Veeam. Little search at Veeam's Forums and I found this topic: http://forums.veeam.com/veeam-backup-replication-f2/can-snapshots-slow-down-running-backup-t22521.html

So, it's quite obvious, if you have snapshots on those VM's that you are backing up with Veeam, it will slow it down because consolidation is slower with nested snapshots. (So, backup itself is still fast, but 'cleaning up' after backup is slower)

But, is it really that much slower?

I had to follow the situation for a while since I was quite curious to see that how big impact does snapshots have. And the impact, it's way bigger than what I suspected.

This table shows impact of removing snapshots from VM's:


VMSnapshot size in GBAverage time to do consolidation with old snapshotsAverage time to do consolidation with just Veeam snapshot
Machine 191 minute15 seconds
Machine 27830 minutes30 seconds
Machine 382 minutesunder 10 seconds
Machine 42512 minutesunder 10 seconds
Machine 5162 minutesunder 10 seconds
Machine 63110 minutesunder 10 seconds
Machine 741,5 minutes15 seconds
Machine 88730 minutesunder 10 seconds
Machine 93415 minutesunder 10 seconds
Machine 102310 minutesunder 10 seconds

Conclusions? Don't keep snapshots on your environment if you don't need them, it will slow down things. And this is just one thing that it slows down, it of course affects to VM performance also, but there are lot of good blog articles about that already. With only 10 VMs, time saved with just removing old snapshots was almost 2 hours. You can just imagine that how much time you can save in you backup jobs if you have ten's or even hundreds of VMs that have (forgotten) snapshots.

And of course, impact is also dependent on your overall storage performance. In this case, underlying storage is quite reasonable, so with slow storage, impact might be even bigger.

Sunday, November 2, 2014

Modifying .XML files with Powershell

I was asked to help on one application upgrade recently, and it required modifying hundreds of XML-files. All changes were similar, so it was perfect place to do some script automation.

Note, this is not the only way to modify XML-files, this is my way :)

How do I modify XML-files with PowerShell? That's exactly the question that I had in my mind. But first, the XML file that I had. This is not the actual file, but it has same elements so we can use this as an example:

<WEBSETTINGS>
   <Locations>
    <Url value="~/Change/needed.html" />
    <LogDirectory value="\Logs" />
    <NotNeeded value="This we need to remove" />
  </Locations>
  <Settings>
    <Version value="1.0.0" />
    <SmtpServer value="smtp.example.com" />
  </Settings>
</WEBSETTINGS>

Now that we have a XML file, how do we read that with powershell? Like this:

$XML = New-Object -TypeName XML
$XML.Load("C:\XML-example\example.xml")

That will read file C:\XML-example\example.xml to variable called $XML. And now we can browse through XMLwith that variable, like you can see in this screenshot:


Wow, that's easy.

Now let's try to modify our XML. And that's as easy as setting a value to variable, like this:

$XML.WEBSETTINGS.Locations.Url.value = "http://scattereditnotes.blogspot.com"

And that's it, we have changed value of attribute "Url".

Then we add totally new attribute to our XML, it's a little bit more work, but still quite easy. So, let's add attribute called 'NewAttribute' with value 'New Value', and we add that under section 'Settings'. And code to do so:

$NewRow='<NewAttribute value="New Value"/>'
$NewXmlAttribute = $XML.CreateDocumentFragment()
$NewXmlAttribute.InnerXml = $NewRow
$node=$XML.SelectSingleNode('//Settings')
$node.AppendChild($NewXmlAttribute)


Now, let's remove one not needed attribute. Since we know exactly what attribute we want to remove, it's easy:

$RemoveAttribute = $XML.WEBSETTINGS.Locations.NotNeeded
$XML.WEBSETTINGS.Locations.RemoveChild($RemoveAttribute)

So, now we have done following things, we read XML-file to variable, modified value of attribute, added new attribute, and deleted one. Only one thing to do, save it.

$XML.Save("C:\XML-example\example.xml")

Whole code is here:


#Read XML-file to variable
$XML = New-Object -TypeName XML
$XML.Load("C:\XML-example\example.xml")
#Modify Attribute
$XML.WEBSETTINGS.Locations.Url.value = "http://scattereditnotes.blogspot.com"
#Add new attribute to section Settings
$NewRow='<NewAttribute value="New Value"/>'
$NewXmlAttribute = $XML.CreateDocumentFragment()
$NewXmlAttribute.InnerXml = $NewRow
$node=$XML.SelectSingleNode('//Settings')
$node.AppendChild($NewXmlAttribute)
#Remove attribute
$RemoveAttribute = $XML.WEBSETTINGS.Locations.NotNeeded
$XML.WEBSETTINGS.Locations.RemoveChild($RemoveAttribute)
#Save changes back to XML-file
$XML.Save("C:\XML-example\example.xml")

Nice, but, my task was to modify hundreds of .XML-files. Files were in directories under main directory, one file per directory. And I also wanted to make a backup file of all the original files.

And here is the final code:
#Get all directories to variable
$Directories = get-childitem|where {$_.Attributes -eq 'Directory'}|select FullName
#Go through all directories
foreach ($Directory in $Directories)
{
 #Read XML-file to variable
 $XML = New-Object -TypeName XML
 $XML.Load($Directory.FullName+"\example.xml")
 #Create backup of file
 $XML.Save($Directory.FullName+"\example.xml.backup")
 #Modify Attribute
 $XML.WEBSETTINGS.Locations.Url.value = "http://scattereditnotes.blogspot.com"
 #Add new attribute to section Settings
 $NewRow='<NewAttribute value="New Value"/>'
 $NewXmlAttribute = $XML.CreateDocumentFragment()
 $NewXmlAttribute.InnerXml = $NewRow
 $node=$XML.SelectSingleNode('//Settings')
 $node.AppendChild($NewXmlAttribute)
 #Remove attribute
 $RemoveAttribute = $XML.WEBSETTINGS.Locations.NotNeeded
 $XML.WEBSETTINGS.Locations.RemoveChild($RemoveAttribute)
 #Save changes back to XML-file
 $XML.Save($Directory.FullName+"\example.xml")
}

This is what our XML-file looks like after our modification.

<WEBSETTINGS>
  <Locations>
    <Url value="http://scattereditnotes.blogspot.com" />
    <LogDirectory value="\Logs" />
  </Locations>
  <Settings>
    <Version value="1.0.0" />
    <SmtpServer value="smtp.example.com" />
    <NewAttribute value="New Value" />
  </Settings>
</WEBSETTINGS>

And that is so beautiful.