Friday, 10 October 2014

Create ZIP Archive in PowerShell and SharePoint IIS and ULS archiving scripts

In my career as Windows Web hosting administrator and now as dedicated SharePoint Administrator I have always needed a way to clean up the IIS logs since the IIS does not provide such functionality.
I guess that this is an issue for everyone involved in some sort of application management that is hosted on IIS.
Most of us leave Log Rollover settings to Schedule, Daily and we end up with bunch of files in our log directory. The size of the logs depends on how many hits the application is receiving. If the application is some public website the logs can become quite big and the volume where the logs are stored  can run out of space (in any case you do not want the IIS logs stored on System drive or on drive with application files).
So, the most recent experience I had was with one of our customers that has their public websites hosted on SharePoint 2013. The sites are quite busy and we have huge 1GB logs almost everyday and with NTFS compression the files are still around 300MB size on the disk.
Since it is a good practice to keep this log files as long as we can, we can have two options.
Option one is to zip(or use our archiving tool of choice) all log files,put it away and delete the original file in order to save some space and option two to write or find a script to do this for us.
I found many scripts for archiving IIS logs. Most of them are using 7-zip to do the zip archives or the cmdlet Write-Zip that comes from the PowerShell Community Extensions. Both tools are using some sort of a 3rd party compiled code that I do not want to run on the servers that are hosting the public sites of our high profile customer.
At the moment we don't have version of PowerShell that has native cmdlets for working with .zip files. The new PowerShell version 5 will have native cmdlets that can create and extract zip files, but it is still in Preview version and it is not recommend for production environments. If you want to check it out there is a preview of the Windows Management Framework V5 available for download.
So, I needed to write my own script to do the work without using any 3rd party tools.
With some research I found some classes that are added to the System.IO.Compression Namespace in .NET 4.5 that might do the work. The .NET Framework 4.5 is a prerequisite for every SharePoint 2013 so I do not need to install anything additional.
Below you can see the function that I have used in the IIS Log Archiving script.


Function New-LogArch{
    [CmdletBinding()]
    Param (
        [parameter(Mandatory=$true)]
        [System.IO.FileInfo]$file,
        [parameter(Mandatory=$true)]
        [string]$Location
    )
    PROCESS{
        Add-Type -Assembly System.IO.Compression.FileSystem
        $tmpFolder = $Location + '\' + $File.BaseName
        New-Item -ItemType directory -Path $tmpFolder | Out-Null
        $destfile = $Location + "\" + $File.BaseName + ".zip"
        $logPath = $file.FullName
        $zippedName = $file.Name
        $compressionLevel = [System.IO.Compression.CompressionLevel]::Optimal
        [System.IO.Compression.ZipFile]::CreateFromDirectory($tmpFolder,$destfile,$compressionLevel, $false )
        $archive = [System.IO.Compression.ZipFile]::Open($destfile, [System.IO.Compression.ZipArchiveMode]::Update )
        [System.IO.Compression.ZipFileExtensions]::CreateEntryFromFile($archive, $logPath, $zippedName, $compressionLevel) | Out-Null
        $archive.Dispose()
        Remove-Item $tmpFolder -Force

    }
}


The function takes two parameters, the information for the file where it can get the literal path to the log file and the location where the archive will be saved.
The first thing we do with the System.IO.Compression classes is to define the compression level in the variable $compressionLevel, I am setting Optimal in order to get smallest file size possible.
Next thing is to create an empty zip archive from a temporary, empty directory I have created earlier. I was unable to find how to create the archive directly from the log file.
When we have the zip file created with the correct Location,Name and Compression Level I "open" it in the variable $archive.
Once we have the archive file opened we use the method CreateEntryFromFile in class System.IO.Compression.ZipFileExtensions, with this method we can add a file to zip archive and we add the original log file to the archive.
Finally we invoke Dispose() method on the $archive, because when we open the zip archive with Update argument the file is locked.
Now when I have the key component that will let me zip the log files I created the IIS log Archive and Cleanup script.
It will find the log location of all IIS sites, will check when they are modified and if they are older than the retention period that you have defined as parameter it will archive the log in the destination you have specified and will delete the original file. You will have a zip file with the name of the original log file that contains only one log file. The script will also check for old archive files and if they are older than the value you have specified it will delete the zip file.
I also wrote a script that can archive the SharePoint ULS logs, because it is a good practice to have them in case you need to do performance analysis and troubleshooting for example.



Download "IIS Log Archive and CleanUp Script" from: Technet Gallery

Download "ULS Log Archive Script" from: Technet Gallery

2 comments:

  1. ULS archive script is giving below error

    Cannot find an overload for "op_Subtraction" and the argument count: "2".
    At E:\Scripts\ULS_Achive\New-ULSLogArchive (2).ps1:113 char:3
    + $timeSpan = ((Get-Date) - ($logDayTime))
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : NotSpecified: (:) [], MethodException
    + FullyQualifiedErrorId : MethodCountCouldNotFindBest

    Exception calling "ParseExact" with "3" argument(s): "String was not recognized as a valid DateTime."
    At E:\Scripts\ULS_Achive\New-ULSLogArchive (2).ps1:112 char:3
    + $logDayTime = [System.DateTime]::ParseExact($logDay, 'yyyyMMdd', $null)
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : FormatException

    ReplyDelete
    Replies
    1. Hi Yagya,

      Can you give me example log name from the farm? It seems that there is an issue during the log time parsing.

      Delete