Cleanup of a Log File or Report Directory

Posted: March 13, 2013 in File Operations, Scripts

Recently had an issue with a poorly coded in house application that was dumping out small report files for every user request and had no cleanup or folder structure for these reports. The end result being some 1.6 million small files in a single directory, all with the same naming structure “FTGReport_. Now, add to this that it is running on Windows 2003 server and you have a problem. As 2003 was still generating short names for files, if you have more than 300,000 files with the same first 6 characters, you hit this odd issue, where file system performance dies a horrible death, as the kernel is looping trying to find a unique shortname to use.

To get around this, I had to attach the VMDK to a Windows 2008 box and use the script below to reorganise all the files into a date stamp based folder hirachy. Because despite the application not actually using any of these files, they needed to still be accessible for historical and audit purposes in some fashion!

# Set the base Path
$BasePath = "D:\Reports"

# How many days of files should be retained in the base path?
$RetentionDays = 60

# Enumerate the list of files that need to be moved
$OldFiles = Get-ChildItem $BasePath | Where {$_.LastWriteTime -lt (date).adddays(-$RetentionDays) -And $_.Mode -NotMatch "d" }

# Cycle through each file
ForEach ($File in $OldFiles){

# Setup the date stamp variables
$LastWrite = $File.LastWriteTime
$Year = Get-Date $LastWrite -Format "yyyy"
$Month = Get-Date $LastWrite -Format "MM"
$Day = Get-Date $LastWrite -Format "dd"

# Create the subdirectory if it doesn't already exist
If (!(Test-Path $BasePath\$Year\$Month\$Day)){
# Using the -force option will create any missing sub directories to complete the full path
New-Item -Path $BasePath\$Year\$Month\$Day -Type Directory -Force
}

# Move the file into its location
Move-Item $File.FullName $BasePath\$Year\$Month\$Day\
}

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s