Hello Readers!

Well, I finally made it…the last post of the year.  I have to say I didn’t think I would be able to make my deadline every week, much less through the entire year.  And with the exception of one week (which screwed me up because of a time change), I managed to pull it off.

So here we have the last script of the year.  This again is another simple script I created.  Most of the time (like I’m sure a lot of forensic examiners do), I like to take the memory images I create and run strings against them to pull out all the text in the raw file (I use the Sleuth Kit srch_string command specifically).  From there I’ll end up running it through grep for possible keywords of interest.

However there are times when I’m on a windows box where I can’t do that as easily.  Yes I could install Unix tools on Windows, but then I wouldn’t be able to write something in Python would I?

What I do with this script is feed in the memory image string file as the input, and then it prompts me for the search term.  Then it goes through the text file and returns the search hit and the line number from the file.  I wanted it to include the line number, so if there’s something interesting I want to look at further, I can open the file in a text program and look at what may be around it.  Of course sometimes the text file is too big for a standard text editor, but at least I have a starting point.

Now where do I go from here?  Well I’m planning on continuing to write code.  I’m going to be taking some of the scripts I’ve used over the last year and expand them out.  One of them being a case management system, something I’ve wanted to write for a while.  Now that I’m not on a weekly schedule I can’t take more time to write code.  I’m also planning on putting together a presentation on my experiences over the last year, which I’ll probably submit to one or two conferences.

I’ll end this last one not by saying until next week, but Keep Coding!


Hello reader!

This week we continue working with the Windows DNS script from last week.  The next step in my process was the ability to search the data.  Now in my case, I’m only interested in two fields, the Remote IP field and the Question Name field.

Mind you this is designed around the script from Week 50, so it’s making assumptions on the table name in the database.

All I’m doing this week with the script is loading the database and then asking the user what they want to look for (IP or DNS name).  Depending on their choice, I run a specific function with the appropriate search term and print the rows with the results.

The one caveat with the DNS search terms, is that you have to end it with a period.  That’s how the data is listed in the database, so if it’s not there you won’t get any results.

Of course for anyone who would like to tailor this script to their needs, you can modify the functions to only return the fields from each row that you are actually concerned with.

Until next week!


Hello Reader!

This week we’re going back and revisiting my Windows DNS script from a couple of weeks ago.  What I had planned to do when I first wrote the script is add the ability to write it all to a SQLite database.  After that I’m going to make a script that will let me search for data within the database.

So the main difference between this script and the previous version is one section:

if os.path.isfile(args.output_file):
    print "{} exists...".format(args.output_file)
    db_connect = sqlite3.connect(args.output_file)
    dns_db = db_connect.cursor()
    # create table
    dns_db.execute("CREATE TABLE dns_record (ID INTEGER PRIMARY KEY AUTOINCREMENT, dns_date text, dns_time text, dns_am_pm text,dns_thread text, dns_context text, dns_ipi text, dns_udp_tcp text, dns_send_recv text, dns_ip text, dns_xid text, dns_query text,dns_opcode text, dns_flagsh text, dns_flagsc text, dns_response text, dns_ques_t text, dns_ques_n text);")

with open(args.backup_file, "r") as new_file:
    for line in new_file:
            dns_record = dns_record_parse(line)
            dns_db.execute("insert into dns_record values(NULL, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?);",(dns_record[0], dns_record[1], dns_record[2], dns_record[3], dns_record[4], dns_record[5], dns_record[6],dns_record[7], dns_record[8], dns_record[9], dns_record[10], dns_record[11], dns_record[12], str(dns_record[13]),dns_record[14], dns_record[15], dns_record[16]))
        except Exception, e:
            print "{}: {}".format(str(e), dns_record)

What we’re doing here is checking to see if the SQLite database exists, if it doesn’t we’ll set up the table to store the data.

Once that part is done, we read in the DNS log file and insert it into the database.  If we run into any issues we print out an error.

Until next week!


Hello Reader!

This week we continue on to part three of parsing out a Norton NPE log file.  For this weeks script, I wanted to focus on parsing out the File details section of the Suspicious Items section, eventually applying it to the Infection Detection section.

I’m dealing with this in two parts, again because of how the XML data is structured in the file.  First is getting to the data to parse, then I need to loop through all the File entries and pull the data out of each.

This week is step one of that process, just pulling the data out of the File structure…

In order to do that, we have to do a for loop in a for loop.  The first loop does the search for the Suspicious Items section.  Once we find that area, we then look “inside” of it for the Files section beneath it.  Once we get to THAT section, we start pulling out the different pieces of data (name, file size, hashes, etc).  Just like last week, I’m putting the data into a dictionary, and then when I go to print it out I can just pick and choose which items I want to display.

For the output, I’m not printing out all the fields, just the particular ones I’m interested in.  Of course anyone is welcome to add/remove whatever they like.

One note to be aware of, I’m not including the Product Name field in the data I collect into the dictionary.  The basic reason is there are instances where you’ll run into Unicode characters (the copyright logo for example).  Since I’m not as concerned with the Product name for this, I’m just not including it in the dictionary I’m creating.

Until next week!


Hello Reader!

Well, we’re coming into the home stretch.  It’s week Forty Eight, and only four weeks left on the calendar.  To be honest I’m shocked I made it this far (and there were some close calls).  But the year isn’t over, so on to this weeks bit of code!

I’m continuing to look at parsing out Norton NPE Log files.  As I stated last week, the logs are in XML format, but not in a “normal” structure that makes it easy to parse with the XML modules.  So I’ve been using BeautifulSoup to tackle this particular issue.

This week I focused on the Infections Detected and Suspicious Items sections of the log file.  But right now I’m only looking at the summaries for these sections.  That way (at least right now) I can go into the log file and target anything that doesn’t have a zero value.

The good news is both sections are parsed out the same way, so once I got one section working, it was just a matter of setting up the second section.

Now what I do in each section is start by looking for the initial section name in the document.  Once I have that, I walk through each of the “children” in this section, and create a dictionary with the values.  The trick with this section is that each child has a line that looks like this:

<DRIVERS Count=”0/>

And if I tell the script to just pull the drivers tag, it pulls the entire line.  But I want to separate out “drivers”, “count”, and “0” when I display the output.  In the dictionary I’m creating, I’ll get “drivers” as the key (using this example), and “count : 0” as the value.  So the value of the dictionary ends up being a dictionary itself, with “count” as the key, and “0” as the value.  This means when I display the output, as I’m walking through the dictionary, my value from the first dictionary is actually value[key], in order to produce the value from the “inner” dictionary.  This was the one part of the script that I had the biggest challenge figuring out.

Until next week!


Hello Reader!

This weeks post is going to be a quick one.  It’s been a crazy week with some personal items I’ve been dealing with.  Which has consisted of sick family members, SNOMGPOCALYPSE, and a 24 hour power outage in my home.  So unfortunately I didn’t get as far with this weeks script as I had hoped, but I have a starting point to work on for week 48.

This week I decided to play around a bit with XML files.  Specifically the log files used by Norton’s NPE utility, which is similar to McAfee Stinger.  At work this week I needed to review some NPE log files, and while it’s easy enough to open it in your favorite browser and navigate through, I thought it would be a nice thing to apply some Python to.

In my research on this script, I started looking at the XML modules that Python already has.  I was struggling a bit with this, because the NPE log files don’t follow a standard format that I was able to utilize very well with the modules out there.  I could get some of the data out of the file, but there were nested tags within the log file that I was having…challenges parsing out.

Then I ran into a post (on Stackoverflow of course) where someone suggested using BeautifulSoup to parse out the data.  I had wanted to do some more work with this module earlier in the year, but it didn’t pan out.  But now I had a good reason to use it.

Which brings me to this weeks script.  All we’re doing at this point is pulling some information out of the log file on the OS of the system it was run against, and then we go through the entire file and print out all the File tags within.  At this stage I was mainly testing out how to parse the data, while I think about the best way to organize and display it.  So there’s not much to it at this point.  However, it does give me the ability to do a quick review of the files listed, and see if there’s anything out of place.

Until next week!


Hello Reader!

This week we continue our Windows DNS log parsing script.  Most of this script consists of the functions I created to parse the data.

First up is the function that parses out the Opcode field of the log.  This field has four possible options, and I’m just taking the value in the log and then returning what it means.  As always the easiest way to store this information is in a dictionary.

# This parses the opcode value
def dns_opcode_parse(opcode):
    dns_opcode = { "Q" : "Standard Query",
                   "N" : "Notify",
                   "U" : "Update",
                   "?" : "Unknown"}
    opcode = dns_opcode[opcode]
return opcode

The next function in this script parses out the DNS Flag Character codes in the log file.  Again I’m following the same formula as before, using a dictionary to track the possible values.  However, the added trick here, is you can have more than one value in that field.  So what I’m doing is breaking out the different characters in the field, looking them up in the dictionary, and then returning all the values.

# This parses the DNS Flag Character codes
def dns_flags_parse(flag):
    dns_flags = { "A" : "Authoritative Answer",
                  "T" : "Truncated Response",
                  "D" : "Recursion Desired",
                  "R" : "Recursion Available"}
    flaglist = list(flag)
    flag_values = []
    for flag_letter in flaglist:
return flag_values

Next up is a function to deal with the DNS question name, the last field in each line of the log.  I was looking to make it easier to read, so I’m removing the ()’s and the numbers contained within, and replacing them with periods.  When I return the result, I’m stripping off the first period that comes at the start of that field.

# This removes the (#) from the hostnames and replaces them with periods
def dns_question_name_parse(dns_name):
    fix_dns_name = re.sub('\(\d+\)', '.', dns_name)
return fix_dns_name[1:]

Finally, the last function goes through and parses each line of the log file.  This was tricky to put together, and it may not catch everything.  The issue with the log is that in some cases, the number of fields per line changes based on the DNS Query Response field.  It only has two values, being blank or a “Q”.  To determine if the field is blank, I have to see if that field is a “Q”, if it is then I’m assuming the Query Response code value is blank, so I have to shift everything over one.

I also have to look at the Response Code value.  Based on what that is determines what fields may be in the remainder of the line.

(I’m not posting the snippet of this section of code since it’s so long)

And then I wrap up the script by going through the log, parsing out the fields, and then writing it to a CSV file.

Until next week!


Hello Reader!

This weeks script comes from a case I’m working on right now for my job, and while I didn’t HAVE to write a python script for it, when I was talking with my co-worker about looking through some Windows DNS logs, he said to me…”There’s your python script for the week.”

And thus, YOP Week Forty Five was born!

The case I’m working on currently revolves around reviewing DNS logs for some specific information.  The Windows DNS logs themselves aren’t that bad to look through, until they are several hundred/thousand/billion megs in size…then you want to create some way to search through it rather easily.  I was starting to look through the data with Notepad++ when I realized it would be a lot nicer if there was an easy way to filter the data down.

So for this week, my script begins the process of what I want the end result script to be.  I did some looking online and there are a few Python scripts out there that parse Windows DNS logs, but they were designed for specific purposes, and I was looking to add some flexibility to mine.

But for this first stage, my goal was to clean up the file, and then output it to CSV format so I can filter it in Excel.  My plan for the next stage is to allow you to search, and clean up some of the formatting errors that are present in this version.

The first part of this script takes the DNS file and runs it through two stages, the first gets rid of the 34 lines at the start of the file.  These lines define the fields in the report.  My plan is to add them in as a header row in the second run through of this script.  The second function gets rid of the new lines in between each of the DNS log entires in the file itself.  For this part I actually create a backup file, since during some of my earlier testing I removed ALL the contents of the file.  Again in the second version I’ll move the backup portion to the first function, so that they entire script works off the backup file instead of the original file.

Finally I walk through the script and split the fields up to output into a CSV file.  The downside to Windows DNS logs is that they are field separated by spaces.  Where that becomes a problem is when you get to the Opcode field.  Since that field can have more than one value, you can have spaces between them, which the script interprets as two separate fields (even through it’s not).  I’m still trying to figure out the logic on that one, so you’ll need to be careful with the final CSV output, you might have some fields that have shifted over on accident.

Like I said this is the first pass on this script.  There are a few more things I need to tweak and fix.  But it’ll be enough to get me started with what I need it for on this case!

Until next week!


Hello Reader!

Ok this is going to be a quick post.  For this week I started working on a script to help my kids practice their division.  It’s similar to the multiplication one I did back with Week 41.  However, what I wanted to do was avoid any remainder problems.  So what I did this time around was generate two random numbers (based on input), multiply those numbers, take the product, and then use that as part of the division problem.  That way it focuses more on what my kids are working on.

That’s it!  Like I said this one was going to be quick….

Until next week!


Hello Reader!

This week I’m changing gears a bit.  Recently I’ve been doing a lot of checking on IP addresses that show up in logs, memory dumps, alerts, etc.  And I decided it would nice to have a simple script that can just pull that information for me automatically without typing multiple commands over and over.

This script takes an IP address and a DNS server as arguments, does a DNS lookup on the IP, then pulls the whois information, and finally will pull down the main webpage from the site (I’m assuming here there’s a webserver on the other end).  The webpage gets saved in the directory the script is run from.

This script does require the following modules in order to run:

ipwhois and pycurl

But otherwise that’s it!

Until next week!