New in Veeam Backup & Replication v10: Linux Repositories with XFS support

One of enhancements in Veeam Backup & Replication v10, is possibility to use XFS file system and FastClone, which is similar that Veeam has utilized with ReFS for years already.

But how to set-up it? First of all, you need to have Linux server that meet's requirements.

Basically what you need is:

• Ubuntu 18.0.4 or later. For other distributions, Fast Clone support is experimental.
• XFS file system
• Cyclic redundancy check (CRC) is enabled
• The minimum supported data block size is 1 KB. The maximum supported block size is 4KB

And you can format your disk like this:

mkfs.xfs -b size=4096 -m reflink=1,crc=1 /dev/sdb1

This is how it looks:

Now we can add our repository to Veeam Backup & Replication, in console, go to Backup Infrastructure -> Backup Repositories -> Right click -> Add backup repository.

(Wan't to do it with PowerShell? Scroll to end of page)

Select "Direct attached storage"

Select Linux

Give name and description for your repository

If you have already added your Linux machine as managed server to Veeam, you can just select it from drop down menu, if not, click "Add new"

Enter FQDN or IP-address of Linux machine. Notice, that you need to have SSH and Perl installed.

Select credentials to use, typically it is best to use dedicated service account. If you have not previously entered account info, you can do it with "Add..."

Validate SSH key fingerprint

And in a short moment you see that server has been added

Now back in "New Backup Repository" wizard, your newly added server will be automatically selected.

Click browse, to select path that is used for this repository

Select folder..

Select "Use fast cloning on XFS volumes" checkbox

Select mount server. Notice, that you cannot use Linux repository as a mount server, it has to be a Windows machine. I'm using my Backup & Replication machine in this example.

You can review your settings

And when you click Apply, your new repo will be added

Finally you will see a summary page that confirms that new repository has been successfully created

We can, of course, do the same thing by using PowerShell, with just couple of lines.

First we get our credentials and then add Linux machine to Backup & Replication. If you already have Linux machine added, you can skip this step. As you can see, I was a bit lazy, and used screenshot from previous post, since process of adding Linux machine does not differ, if it's used as a Proxy or a Repository

Then we configure it as a repository. As you can see, we first read information about mount server and Linux server to variable, and then use them as parameters when we add repository.

You also need to set folder, type and use "-EnableXFSFastClone" to enable Fast Cloning in XFS volumes.

Actual code:

$credential = Get-VBRCredentials -Name <YOURCREDENTIALNAME>
Add-VBRLinux -Name <FQDN OF LINUX MACHINE> -Credentials $credential
$mountserver = Get-VBRServer -Name <YOURMOUNTSERVER>
$reposerver = Get-VBRServer -Name <YOURREPOSERVER>
Add-VBRBackupRepository -Name <YOURREPONAME> -Description <YOURDESCRIPTION> -Server $reposerver -MountServer $mountserver -Folder <YOURFOLDER> -Type LinuxLocal -EnableXFSFastClone

New in Veeam Backup & Replication v10: Linux Proxies

One of the many new features in Veeam Backup & Replication v10, is Linux proxy.

You can use Linux proxies in your VMware environment.

Adding new Linux proxy is really simple.

(Wan't to do it with PowerShell? Scroll to end of page)

In Backup & Replication console, go to Backup Infrastructure -> Backup Proxies -> Right click -> Add VMware backup proxy.

If you have added your Linux machine as a managed server to Backup & Replication, select it from drop down list, otherwise select "Add New..."

Select "Linux"

Enter FQDN or IP-address of Linux machine. Notice, that you need to have SSH and Perl installed.

Select credentials to use, typically it is best to use dedicated service account. If you have not previously entered account info, you can do it with "Add..."

Validate SSH key fingerprint

And in a short moment you see that server has been added

Now back in "New VMware proxy" wizard, your newly added server will be automatically selected.

Do note, that "Transport mode" and "Connected datastores" selections are grayed oyt. That is, because Linux proxy only works in "Virtual appliance" mode

If in VMware VM settings, you do not have disk.enabledUUID=TRUE parameter, you will get following warning. Select "Yes" to pick VM from vCenter manually

Select "Browse..." to select your VM

You can use search to find your machine

And you will see your machine selected

You will also get a warning that your Linux machine will be rebooted. This is because Veeam add's that missing parameter to VM's configuration

You can set Throttling settings if needed

And when you click "Apply" your proxy will be configured

And finally you get summary that states that Linux Proxy has been successfully created

And that's it! Your new Linux proxy is now ready to be used in backup jobs. If you are using automatic selection in job settings, it will automatically be used in next backup and replication jobs!


Looking for PowerShell magic? Well, here we go:


First we get our credentials and then add Linux machine to Backup & Replication. If you already have Linux machine added, you can skip this step

Same as with wizard, if you VM does not have disk.enabledUUID=TRUE parameter in place, you will get error when you try to add Linux proxy:

So we will first search our VM, and give it as a parameter to PowerShell command:

Code:

 

$credential = Get-VBRCredentials -Name <YOURCREDENTIALNAMEHERE>
Add-VBRLinux -Name <FQDN OF LINUX MACHINE> -Credentials $credential

$vientity = Find-VBRViEntity -Name "<VMNAME IN VSPHERE>"
Get-VBRServer -Name "<FQDN OF LINUX MACHINE>" | Add-VBRViLinuxProxy -ProxyVM $vientity

Using F-Secure Server Protection with Veeam Secure Restore

Do note, this is not a "best practice" guide, rather an example how I did it in my homelab, and it demonstrates how easy it is to integrate AV scanner to Secure Restore feature in Veeam Backup & Replication, Update 4


Veeam introduced feature called "Secure Restore", with version 9.5 Update 4 of Backup & Replication, that was released on January 2019.

It allows you to scan machine data with antivirus software before restoring it to the production environment. And you can also use this when testing your backups with Sure Backup.

You can integrate basically any AV software with Veeam, as long as your AV supports command-line scanning.

How Secure Restore works, and how to integrate AV software, is documented here: https://helpcenter.veeam.com/docs/backup/vsphere/av_scan_about.html?ver=95u4

But I wanted to test how it's actually done. So I got my a self a trial version of F-Secure Server Protection. It's part of their cloud managed "Protection Service for Business", aka. PSB.

I installed that AV product to one of my Backup Repository servers, that also acts as a mount server.

To configure Veeam integration, I need to modify AntivirusInfos.xml file, that is located under "%ProgramFiles%\Common Files\Veeam\Backup and Replication\Mount Service" folder.

After reading documentation, I knew that I need at least path to AV scanner, required command line parameter(s), exit codes and a string that is shown when infection is found.

First, I started to figure out that how command line scanning works. I found out, that there is a fsscan.exe, located in "C:\Program Files (x86)\F-Secure\PSB\" directory. When you run it, it show's you command line parameters.

With this, I knew that I need to use --target parameter, to specify what to scan.

For exit codes, I found this page: https://community.f-secure.com/t5/Business-Suite/On-demand-scanner-fsav-exit/ta-p/20254 that shows exit codes for fsav.exe, but I did not find anything for fsscan.exe. But since fsav.exe is old name for fsscan.exe, I decided to test if those exit codes still work.

I then downloaded EICAR test file, to test what happens when I manually run command line scan. (It's a bit tricky to download these days, you have to disable all your AV protection to be able to download it).

First I tested scanning with folder that had no infected files in it, to see how it looks.

After that, I copied EICAR test file to couple of folders, among with other clean files and run a test again.

When comparing clean and infected results, I searched for a unique string that I can use to detect when infection is found. That string seems to be "Infections found.", with a dot (".").

So now I had a bare minimum information I needed to add to configuration XML file. Since I wanted to to this with minimum effort, I decided to use "IsPortableSoftware='true'" parameter, so I can leave "RegPath='' ServiceName=''" parameters empty.

I used these parameters:
AntivirusInfo Name='F-Secure Server Protection'
IsPortableSoftware='true'
ExecutableFilePath='C:\Program Files (x86)\F-Secure\PSB\fsscan.exe'
CommandLineParameters='--target %Path%'
RegPath=''
ServiceName=''
ThreatExistsRegEx='Infections\s+found\.'
IsParallelScanAvailable='false'

And also added ExitCodes, which you can see from full code snip.

My addition to XML files looks like this:

 
<AntivirusInfo Name='F-Secure Server Protection' IsPortableSoftware='true' ExecutableFilePath='C:\Program Files (x86)\F-Secure\PSB\fsscan.exe' CommandLineParameters='--target %Path%' RegPath='' ServiceName='' ThreatExistsRegEx='Infections\s+found\.' IsParallelScanAvailable='false'>  
           <ExitCodes>  
                <ExitCode Type='Success' Description='No threats detected'>0</ExitCode>  
                <ExitCode Type='Error' Description='Fatal error; unrecoverable error.'>1</ExitCode>  
                <ExitCode Type='Infected' Description='Virus threat was detected'>3</ExitCode>  
                <ExitCode Type='Infected' Description='Riskware (potential spyware) found'>4</ExitCode>  
                <ExitCode Type='Warning' Description='Suspicious files found; these are not necessarily infected by a virus'>8</ExitCode>  
                <ExitCode Type='Error' Description='Scan error, at least one file scan failed'>9</ExitCode>                 
           </ExitCodes>  
      </AntivirusInfo>  

After adding those lines to my AntivirusInfos.xml, I started doing some tests.

I created some dummy virtual machines, one with EICAR test file, one with "Potentially Unwanted Application" test file from AMTSO, and one clean machine. All those machines are dummy machines with no operating system, there is only a 4GB disk in each of those machines with some random files in there.

To test, I started full VM recovery for all of those three test machines. Only thing I need to do while restoring, is to check Secure Restore option:

I set it to abort restoring, if infection is found. With both F-Secure and Microsoft Defender, they do not report you about found infections during AV scan with a string that could be parsed, so it always does a full scan, so "Scan entire image" check box does not have effect with those two AV scanners.

And how does it look like? With a clean machine, it looks like this:

Everything is nice and green, and restore was successful.

But with infected machines, it looks different. Both EICAR and "Potentially Unwanted Application" are detected:

I was expecting that with "Potentially Unwanted Application", F-Secure would have used Exit Code 4, but it seems to detect it as an infection instead of riskware.

I also tested those same machines with Microsoft Defender. I expected to get same results, but I did not! With clean machine, and machine with EICAR test file, results were the same, but with "Potentially Unwanted Application", Microsoft Defender did not detect anything harmful! I'm not sure if this is some setting error in my servers MS Defender or not, but quite a worrying anyways. To be sure, I actually tested it with two different servers, and with same results.

As a last thing, I wanted to see if there is any performance difference between F-Secure Server Protection and Windows Defender.

So I restored a real Windows 10 machine, that has some EICAR files in it.

With F-Secure Server Protection, it lasted about 10 minutes:

And with Windows Defender, it lasted a bit over 20 minutes:

So, actually quite a big difference.

As you can see, it's really easy to integrate you AV scanner to be used with Secure Restore functionality in Veeam Backup & Replication!

Connecting vRealize Orchestrator 7.6 to vCenter

Do note, this is not a "best practice" guide, more a documentation on how I did configuration in my home lab

Most likely the first thing that you want to do with your Orchestrator, is to add your vCenter instance to your Orchestrator, so you can run workflows against it.

To do that, let's log in to your Orchestrator HTML5 Client.

Go to workflows. We need to search the right workflow for us. Use two keywords, "add" and "vcenter", and we should be able to find a correct workflow

Click "RUN" on "Add a vCenter Server instance" workflow to start it.

Give your vCenter IP or FQDN to form. You might also want to check "Do you want to ignore certificate warnings? If you select Yes, the vCenter Server instance certificate is accepted silently and the certificate is added to the trusted store", at least if you are using self-signed certificates.

Go to "Set the connection properties" tab, and give username and password that are used to connect to vCenter. 

Click "Run", and wait for workflow to run.

If everything goes as expected, you should see that status is "completed".

You can now go to Administration -> Inventory, and you should be able to see your vCenter inventory from here:

We have now succesfully added our vCenter instance to Orchestrator!