ZFS forensics scrollback script
 What is this?
This is a python script that enables to get back already deleted files or partitions. This is highly experimental, but I managed to get back a moths work when all the partitions were deleted and created new ones on top by accident. This will only work if the pool is not very full and there was not very much activity after the unwanted action.
 How it works
- The script finds all uberblocks, reads their metadata and orders them by time
- Enables you to destroy all the uberblocks that were created after the event that you want to scroll back to.
(Uberblocks are created every time data gets edited in ZFS. Every uberblock creates a new "path" to the data, and the new path includes data changes, by deleting the last uberblocks, ZFS will use the last "path" witch leads to the old data.) This might not be 100% true, but this is how i understand it. Shortly ZFS amnesia.
- It might be that reverting to a TXG where the pool was in an exported state, then there is a better chance of importing it after.
- Don't know how to do this with multiple disc arrays. But i think you just have to delete the uberblocks from all the discs. Could someone test this please?
- A zpool recovery feature that rolls back several transactions integrated into SXCE release, build 128. You can read about this feature here:
 Example walkthrough
#lets create a 500mb file. mkfile 500m test_filesystem #mount it as a lofi device that will be called /dev/lofi/1 lofiadm -a test_filesystem #now we make a pool. zpool create testpool /dev/lofi/1 #lets just verify all that. zpool list NAME SIZE USED AVAIL CAP HEALTH ALTROOT ... testpool 492M 73K 492M 0% ONLINE - ... #i will make 2 different filesystems #later i will delete some files and write new and delete the other completely. zfs create testpool/a zfs create testpool/b zfs set mountpoint=/export/home/mardicas/TEST/a testpool/a zfs set mountpoint=/export/home/mardicas/TEST/b testpool/b #i will use some homevideos for testing du -sh sourcefiles/* 140M sourcefiles/2009.05.07 232M sourcefiles/2009.05.08 #lets copy them. cp sourcefiles/2009.05.07/* a/ cp sourcefiles/2009.05.08/* b/ #make sure they are written. sync #i will export and import the pools to make it easier to find the right place later. zpool export testpool zpool import -d /dev/lofi/ testpool #now lets do some damage!(you can even copy new files on top) rm -rf a/* zfs destroy testpool/b #well lets copy something else there now... cp /etc/* a/ sync #OOPS i should not have deleted those videos, not to mention the entire testpool/b. #(now i follow the procedure described in the How-to) zpool status testpool zpool history -il testpool dd if=test_filesystem of=test_filesystem.bu ls -l test_filesystem -rw------- 1 mardicas staff 524288000 2010-01-10 20:40 test_filesystem #524288000/512=1024000 #make zfs deal gracefully with errors! #This is probably ignored later because we delete the uberblocks and ZFS will get an Amnesia. zpool set failmode=continue testpool #run the script ./zfs_revert-0.1.py -bs=512 -tb=1024000 test_filesystem #when we printed out the history earlier there is a line: #2010-01-10.20:31:31 zpool export testpool [user root on opensolaris:global] #this is where we would like to be! #the program returned the uberblocks ranging from 4-146, and we will delete all of them after #TXID DATE Timestamp Uberblock locations(4 of them) #103 10 Jan 2010 20:31:31 1263148291 [462, 974, 1023438, 1023950] rm -rf /etc/zfs/zpool.cache #Now i reseted the machine! (brutally) Reboot might not work here... lofiadm -a test_filesystem #lets import the filesystem and tell it to handle errors gracefully(again) zpool import -d /dev/lofi/ testpool && zpool set failmode=continue testpool zpool list NAME SIZE USED AVAIL CAP HEALTH ALTROOT ... testpool 492M 373M 119M 75% ONLINE - ... zfs list NAME USED AVAIL REFER MOUNTPOINT testpool/a 140M 87.5M 140M /export/home/mardicas/TEST/a testpool/b 232M 87.5M 232M /export/home/mardicas/TEST/b mkdir restored mkdir restored/a mkdir restored/b cp /export/home/mardicas/TEST/a/* restored/a/ cp /export/home/mardicas/TEST/b/* restored/b/ /export/home/mardicas/TEST/b/m2u00794.mpg: I/O error #seems like we did lose one file. #Probably because we did copy file to A and they were written somewhere where the m2u00794.mpg file was before #lets make sure everything is like it was before! diff -r restored/a/ sourcefiles/2009.05.07/ #looks good :-) diff -r restored/b/ sourcefiles/2009.05.08/ Only in sourcefiles/2009.05.08/: m2u00794.mpg #well i was asking for it :P
First check the pool status:
zpool status zones
Get the disc name e.g:c2t60060E800457AB00000057AB00000146d0
Now we look up the history of the pool so we can find the timeline and some uberblocks(their TXG-s) where to scroll back:
zpool history -il zones
Save this output for later use.
You will defently want to backup the disk before you continue from this point:
ssh root at host "dd if=/dev/dsk/c..." | dd of=Desktop/zones.dd
The script accepts 2 arguments:
-bs is block size, by default 512 (never tested) -tb is number of blocks:[this is mandatory, maybe someone could automate this]
- To find the block size in solaris you can use
prtvtoc /dev/dsk/c2t60060E800457AB00000057AB00000146d0 | grep sectors
From there look at the "sectors" row.
- If you have a file/loop device just sizeinbytes/blocksize=total blocks
Now run the script for example:
./zfs_revert.py -bs=512 -tb=41944319 /dev/dsk/c2t60060E800457AB00000057AB00000146d0
This will use dd, od and grep(GNU) to find the required information. This script should work on linux and solaris.
Something like this should appear, if not then something is wrong:
TXG, time-stamp, unixtime, addresses(there are 4 copy's of uberblocks) 411579 05 Oct 2009 14:39:51 1254742791 [630, 1142, 41926774, 41927286] 411580 05 Oct 2009 14:40:21 1254742821 [632, 1144, 41926776, 41927288] 411586 05 Oct 2009 14:43:21 1254743001 [644, 1156, 41926788, 41927300] 411590 05 Oct 2009 14:45:21 1254743121 [652, 1164, 41926796, 41927308]
- Take a wild guess witch block might be the one, it took me about 10 tries to get it right, i don't know what are the "good" blocks. The history we printed out earlyer might help!
- Enter the last TXG you want to KEEP.
- The script writes zeroes to all of the uberblocs after the TXG inserted.
- Now clear the ZFS cache and reboot, cross fingers.
rm -rf /etc/zfs/zpool.cache && reboot
- First try to import the pool if it is not imported yet.
- Make sure to set the failmode right after!!! It might freze otherwize.(no use issueing it before, because ZFS has amnesia!)
zpool import -f zones && zpool set failmode=continue zones
Now see if it can import it or fail miserably. There is a good chance that you will hit Corrupt data and unable to import, but as i said earlier it took me about 10 tries to get it right. I did not have to restore the whole disk every time, i just took baby steps and every time deleted some more blocks until found something stable (it will still crash after few minutes, but this is enough time to get back conf files or some code)
 The script
Can be downloaded from http://www.loovsys.eu/redmine/projects/zfsrevert Please send any patched or improoved versions to email@example.com and i will upload them!
Authors: Martin Vool - firstname.lastname@example.org
I got so far by the help of these materials!