Combining multi-line events into single transactions in Splunk

Posted on Updated on

This can be used to group together multiple lines, of which belong to the same thread. You can pipe the data to a “transaction startswith/endswith” as described below, and Splunk will group the lines into a single shared “transaction”. Then the “table” command basically creates a table of the output of the field.

NOTE: This is slow and very taxing for Splunk, so do it against a short time frame, and don’t schedule any reports to do this (the “transaction” function).

source=”/source/to/data” (host=”server1″ OR host=”server2″) | transaction startswith=”starting word” endswith=”ending word” | search field1 | table field2

Advertisements

Joining RedHat Servers to Active Directory

Posted on Updated on

Joining Redhat servers to AD domain
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

yum install samba3x
yum install winbind

vim /etc/nsswitch

=================================
passwd: files winbind
shadow: files winbind
group: files winbind
=================================

vim /etc/samba/smb.conf

===================================
workgroup = DOMAINNAME
password server = x.x.x.x
realm = DOMAINNAME.COM
security = ads
idmap uid = 10000-20000
idmap gid = 10000-20000
winbind separator = +
template homedir = /home/%D/%U
template shell = /bin/bash
winbind use default domain = false
winbind offline logon = false
====================================

net ads join -U user@domainname.com

Backup and Archive Nagios

Posted on Updated on

This is a shell script that can be used to backup a Nagios configuration (or any group of files/directories in Linux or UNIX) locally, and to sync the backups to a remote location. This script works perfectly when there are two different Nagios instances in different locations, and this script can be used on both servers to backup and archive, and then rsync the files to the remote side – just by changing the three variables at the top of the script. Logging and emailing results of each job can be added in to the script as well.

A best practice that I implemented is to use SSH shared keys for the rsync. Use a non-root account and send the traffic along a trusted VLAN. This allows for the SSH to not prompt for a password every time the script is run, which should be automated through a cron job.

#!/bin/bash

####################################
###### Local System Variables ######
####################################

NAGIOS=usr/local/nagios
LOCAL=/local/directory/path
REMOTE=user@server:/local/directory/path

####################################
####### DO NOT CHANGE BELOW ########
####################################

BACKUP=$LOCAL/nagios-backup.tgz
DATE=`date +"%F-%T"`

export LOCAL
export REMOTE

### check to see if current backup file exists ###
if [ -f $BACKUP ]
then
  echo "Backup file exists."
  mv $BACKUP $BACKUP-$DATE
  tar czf $BACKUP -C / $NAGIOS
else
  echo "Backup file does not exist...creating."
  tar czf $BACKUP -C / $NAGIOS
exit
fi

### remove files older than seven days ###
find $LOCAL -type f -mtime +7 -exec rm {} \;

### change the permissions of the file to the backups user ###
chown -R backups:backups $LOCAL

### change to backups user to run the rsync script ###
su backups -c /home/backups/rsync-files.sh

### rsync the files to the DR backup site ###
rsync -avz --delete $LOCAL/ $REMOTE

Script for emailing DFS Replication Health Reports

Posted on Updated on

DFS replication is a great way to synchronize data for DR purposes, but there is no built in scheduled reporting mechanism.  Well here is a script I wrote that runs the dfrsadmin reports and attaches each report, as well as sends links to each report for review.  Very helpful considering I was logging in each day and running commands to check the backlog.  Now, I can just open these reports every day and all the information is right there.  Once the script is there, simply create a scheduled task to run this script at whatever time interval is needed to receive these reports.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

@echo off

set CURRDATE=%TEMP%\CURRDATE.TMP
set CURRTIME=%TEMP%\CURRTIME.TMP
set REPORTS=\\*******\*******
set FROM="DFS Replication <******@*******>"
set TO="******* <*******@*******>"

DATE /T > %CURRDATE%
TIME /T > %CURRTIME%

:: This cleans up old reports to conserve space.
FORFILES /p E:\****************** /m *.* /d -30 /c "cmd /c del @FILE"

:: adds the date/time to the report name and to the title of the email
for /F "tokens=1,2,3,4 delims=/, " %%i in (%CURRDATE%) Do SET DDMMYYYY=%%j-%%k-%%l
for /F "tokens=1,2,3 delims=:, " %%i in (%CURRTIME%) Do Set HHMM=%%i%%j%%k

set RG_Report=%REPORTS%\folder1-%DDMMYYYY%-%HHMM%.html

:: define the report options as specified in the dfrsadmin.exe utility
dfsradmin health new /rgname:folder1 /refmemname:server1 /ReportName:%RG_Report% /fscount:true

:: overwrite the report file names to temp
echo folder1 %RG_Report% > %TEMP%\healthMessageBodyRG.txt

:: include the links to the reports up in the body of the message
echo folder1 %RG_Report% > %TEMP%\healthMessageBody.txt

:: format the individual report to be sent as an attachment
for /F "tokens=2 delims= " %%i in (%TEMP%\healthMessageBodyRG1.txt) Do SET FILESRG=%%i

:: email the links as well as the attachments using sendEmail.exe
sendEmail.exe -f %FROM% -t %TO% -u "DFS Replication Health Reports %DDMMYYYY%" -o message-file=%TEMP%\healthMessageBody.txt -s smtpserver.domain.com -a %FILESRG%

mRemote – one stop shop for server mgmt

Posted on Updated on

If you haven’t used mRemote, I strongly recommend you do.  This is a great application that is a “one stop shop” remote server and device management tool.  You can manage anything and everything which uses these protocols  (RDP, VNC, ICA, SSH, Telnet, RAW, Rlogin and HTTP/S).  So basically any UNIX, Linux, Windows server, any network switch or device that you are remotely administering – all through one application.  The connections can be easily duplicated, and sorted as well as folders for organization.

All in all, a great tool for managing a mixed environment of OS and devices!

Here’s a link to the product overview:

http://www.mremote.org/wiki/Overview.ashx

Target To Start Selling The iPad On October 3rd, Discounts Available (via TechCrunch)

Posted on

Target To Start Selling The iPad On October 3rd, Discounts Available Target will soon be able to fulfill all your iPad needs. October 3rd is the date that the Apple iPad should hit Target stores throughout the US. Best of all, Target credit card holders can get the iPad for a bit cheaper. Target's retail plan includes all six models of the iPad along with a full range of accessories and add-ons. The retailer will honor the suggested manufacturer price starting at $499 for the 16GB WiFi version. … Read More

via TechCrunch

BASH script to Email list of “new” files found

Posted on Updated on

This is a quick-and-dirty script that I made to solve a real world scenario.  I had to find a way to notify a data integration group of any new files that an outside vendor had sent up to our secure FTP SUSE Linux server.  These were batched up into many, many files, all starting with the same few characters.  This made it fairly easy to add a wildcard search, but the other parts deal with the fact that only NEW files needed to be identified, not any existing files that were already processed internally by the data integration group.

#!/bin/bash

OLD=/root/filecheck/old.log
NEW=/root/filecheck/new.log
DIFF=/root/filecheck/diff.log
RCPT=user@user.com

# write an "old" file listing if first time run

if [ -f $OLD ]; then
 ls -la /path/to/filetype/filetype* | awk '{print $9}' > $OLD
fi

# take a snapshot of the directory now, possibly capturing new files

ls -la /path/to/filetype/filetype* | awk '{print $9}' > $NEW
diff "$OLD" "$NEW" >> "$DIFF"

# new file listing now becomes "old" for next run

cat $NEW > $OLD

# if new files are found, log and send out a message

if [ -s $DIFF ]; then
cat <<EOF | /usr/sbin/sendmail -t
To: $RCPT
From: sftp@sftp-server.company.net
Subject: New Files Found
`cat $DIFF`
.
EOF
fi