System Forensics

All your artifacts are belong to us.

9 03 2012

Forensics Journey into Python

sysforensics | Digital Forensics, Parse, Python, sqlite3

I have been super busy lately so I haven’t been able to post much. I have also been spending some time learning about Python, which leads me to this post.

Ok, so I have been kicking around the idea of learning Python for awhile now. Why Python you ask? Quite honestly, I have no idea why Python is/isn’t better/worse than another language. I simply did it because there seems to be a lot of helpful information on the web, and a lot of security people have picked it up as their language of choice. I am a “learn by doing/seeing guy”. So, it helps when I can pull some code down that someone has already done and take a look at how they did it. It also helps that my younger brother knows a bit about Python too. :)

But why not use other peoples programs you ask? Well, I personally believe it is necessary these days to have “some” coding/scripting skills if you want to out perform your peers, and grow as an individual. Another reason is because you waste a lot of time waiting for someone else to write something for you. Networks, Systems, Devices, etc. are just too damn big these days. From what I read the old school forensics of yesterday will not work anymore.

Let’s just take my apartment for instance. How would you manually go through all of this if I committed a crime? I have; 2 Macbook Pros (mine has multiple VMWare instances on them), 2 desktops (Windows), (mine has a myriad of VMWare instances on them), 1 media server  (Windows Server) with around 3 – 4 TB of “stuff”, a smart tv, PS3, Xbox 360, PS Vita, 2 android tablets with additional 32BG SD cards, 1 itouch, multiple old/spare hard drives, an Apple TV and 2 cell phones. Hell…that’s almost impossible to go through even with tools.

You can no longer manually look through a system and find everything.

I’m not saying you can’t excel at your job if you can’t/don’t code, but chances are, if you’re one of the people excelling without coding skills you’re doing it by riding on the backs of other coders. So IMHO there really isn’t an argument for the other side on this one. It’s also coming from someone who can’t code (myself). That’s just my two cents. I get very frustrated at the office at times when I can’t automate certain things because my coding skills lack. I’m also a one man security shop so I don’t have the luxury of turning around and saying, “Hey Bob…code this up for me buddy. I’ll get lunch today”

I have tried MANY times to sit down and learn coding from a book and couldn’t ever do it. I’ve literally tried coding for 3 or 4 years, but I never stick with it. This time I am going to change that.

But how are you going to change that Patrick? Well, 3 – 4 years ago I didn’t know/think much about forensics and now that I am learning more about forensics it’s a perfect opportunity to learn coding by way of forensics. We don’t like to do things we don’t like.

So I was thinking about what to write as my first “forensics program”. While thinking I realized that the forensics community is pretty fragmented. I don’t know what other people are working on so it’s hard to find something that might be useful to other people as well as a great learning experience.

In either case, I decided to write a very simple Google History parser script that more than likely no one will ever use. That wasn’t my intention starting out though. I could care less if anyone used it. Maybe at some point I can contribute a tool that people can use. Right now i’m not even crawling so I sure as hell am not thinking about walking.

The important take away is that I learned a little bit about how Google stores information, I learned a bit about SQL commands, date and time stamps, and I learned a little bit about Python. To me that’s worth its weight in gold and much more important than whether the program I wrote can/will be used by others. I think this is where people shy away from coding. They think, “What can I code that someone is going to use?” Who gives a shit. Do it for yourself. That’s exactly what I am doing and that’s exactly what you should be doing.

So….drum roll….

I call it “goog_history.py”. You can call it whatever you want, and you can even tell your friends that you coded it. Just let me know so I know to cover for you. You can run it via the command line like this:

patrickolsen$ python2.7 goog_history.py History\ Index\ 2012-02 > out.txt

import sqlite3 as lite
import sys
from sys import stdout as out, stderr as err
try:
dbfile = sys.argv[1]
except IndexError:
err.write(“Usage: %s \n” % sys.argv[0])
exit(1)

con = lite.connect(dbfile)

with con:
    
    cur = con.cursor()    
    cur.execute(“SELECT datetime((time/1000000)-11644473600, ‘unixepoch’, ‘localtime’), c0url FROM info, pages_content”)
    
    while True:
      
        row = cur.fetchone()
        
        if row == None:
            break
            
        print (row)

The output looks like this (snip is something I added to this post for personal reasons):
(u’2012-02-01 20:06:47′, u’https://mail.google.com/mail/#drafts/1..snip..b’)
(u’2012-02-01 20:06:47′, u’https://mail.google.com/mail/#drafts/1..snip..a’)
(u’2012-02-01 20:06:47′, u’https://mail.google.com/mail/#drafts/1..snip..3′)
(u’2012-02-01 20:06:47′, u’https://mail.google.com/mail/#drafts/1..snip..0′)
(u’2012-02-01 20:06:47′, u’https://mail.google.com/mail/#drafts/1..snip..c’)
So this wasn’t very useful to anyone. I decided to write a history_parse.py script to make this a little more useful.
You can run this one like this: 
patrickolsen$ python2.7 history_parse.py out.txt > parsed.txt
import sys

try:
pf = sys.argv[1] #pf = parsefile
except IndexError:
err.write(“Usage: %s \n” % sys.argv[0])
exit(1)
data = open(pf, ‘r’)

for each_line in data:

z = each_line.split(“,”)

#choose the index[1] from that above array and strip the ” u’” from the left side
tln = z[0].lstrip(“(u’”)
tln = tln.rstrip(“\’”)

vis = z[1].lstrip(” u\’”)
vis = vis.rstrip()

print (tln).rstrip() + “|” + (vis).rstrip(“\’)”)

data.close()

And here is the output of the parse script.
2012-02-01 20:06:47|https://mail.google.com/mail/#drafts/1..snip..b
2012-02-01 20:06:47|https://mail.google.com/mail/#drafts/1..snip..a
2012-02-01 20:06:47|https://mail.google.com/mail/#drafts/1..snip..3
2012-02-01 20:06:47|https://mail.google.com/mail/#drafts/1..snip..0
2012-02-01 20:06:47|https://mail.google.com/mail/#drafts/1..snip..c
My point of this post is that it’s actually pretty easy to get some pretty useful scripts up and going rather quickly. The best book so far I found was Head First Python (link below). Python 2.6 Text Processing was decent as well. For some reason I liked Head First Python the best. That followed up with some Google searches and looking at other people’s code got me this far…..which isn’t that far, but it’s a start. Now the race is on and i’m on the track running my ass off.

References:
thenewboston - Python Videos


 

Comments are currently closed.

4 thoughts on “Forensics Journey into Python”

  • blog says:
    March 9, 2012 at 7:20 pm

    Nice post, and good luck with your learning.

    If you’re not ashamed to have the world critique your code I found it surprising how many people will gratefully use code that you (the coder) believed to be poor.

    My own code skills are poor (I can write bad code in any language, but beyond that I’m worse than poor), but I regularly have people using, and improving (crowd-sourcing ftw) little scripts I write and share.

    Python also my current weapon of choice.

  • id says:
    March 10, 2012 at 12:39 am

    You might consider modifying the first script to format the output as desired. The type of the variable `row` is a tuple, which is a set of elements that can be accessed by a numeric index. So you can do something like:

    print “%s|%s” % (row[0], row[1])

    This can be further simplified because the string composition % operator takes a tuple as an argument:

    print “%s|%s” % row

    since row is a tuple.

    • Patrick Olsen says:
      March 10, 2012 at 9:44 am

      Great. Thanks for your comment. I will read up on that now and see if I can’t get it modified and working before the weekend is over. I appreciate the time you took to explain it in more detail.

  • dubnsnow says:
    April 10, 2012 at 3:49 am

    I suggest you read the book Python Essential Reference if you are already familiar with basic programming principles.

Take the red pill and follow….

Recent Posts

Archives