Difference between revisions of "Forensics on Incident 4"

Revision as of 20:07, 5 March 2017

Context

External WD drive can't be mounted anymore.

Physically the drive has directly a USB3 interface, no SATA.

The biggest problem is that for some sectors, the drive sends bad data instead of a read fail.
Even the first block (MBR) is sometimes garbage, sometimes ok.
So there is no much guarantee even in the "successfully rescued" sectors.

MBR

sudo fdisk -lu wd.img 
Disk wd.img: 6.8 GiB, 7314894848 bytes, 14286904 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00042ada

Device     Boot Start        End    Sectors   Size Id Type
wd.img1    *     2048 1953458175 1953456128 931.5G  7 HPFS/NTFS/exFAT

So there is a NTFS partition located at offset 2048*512

Loop mount

As MBR fails often, we'll mirror the whole sda and not just sda1 which fails often to be created.
So when we'll try to access the NTFS partition, we'll have to tell the offset to our NTFS tools and, if we want to mount it, we've to create a loopback device at the right offset.

See http://madduck.net/blog/2006.10.20:loop-mounting-partitions-from-a-disk-image/

Check for a free loopback device:

sudo losetup -l
NAME       SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE                                       DIO
/dev/loop1         0      0         1  0 /home/docker/devicemapper/devicemapper/metadata   0
/dev/loop0         0      0         1  0 /home/docker/devicemapper/devicemapper/data       0

So we've to use /dev/loop2

sudo losetup /dev/loop2 wd.img -o $((2048 * 512))

Testing:

sudo file -s /dev/loop2
/dev/loop3: DOS/MBR boot sector, code offset 0x52+2, OEM-ID "NTFS    ", sectors/cluster 8, Media descriptor 0xf8, sectors/track 63, heads 255, hidden sectors 2048, dos < 4.0 BootSector (0x80), FAT (1Y bit by descriptor); NTFS, sectors/track 63, sectors 1953456127, $MFT start cluster 786432, $MFTMirror start cluster 2, bytes/RecordSegment 2^(-1*246), clusters/index block 1, serial number 08a2cf4f62cf4de5f; contains Microsoft Windows XP/VISTA bootloader BOOTMGR

Mounting:

sudo mount -t ntfs-3g -o ro /dev/loop2 mnt

The NTFS filesystem is damaged so this fails.

Removing loopback device:

sudo losetup -d /dev/loop2

ddrescue

Basic usage

sudo ddrescue /dev/sda wd.img wd.img.map

This may be quite heavy on the system, so we can re-nice the IO activity:

sudo ionice -c 3 -p $(pgrep -x ddrescue)

A nice tool to display progress maps:

ddrescueview wd.img.map &

It's possible to zoom with the mouse wheel

Advanced usage

Target an area:

-i start -s size

 ddrescue -i0 -s100M /dev/sda wd.img wd.img.map
 ddrescue -i999G /dev/sda wd.img wd.img.map

Direct access on input, bypassing kernel cache:

-d

Retry 3 times on errors:

-r3

Backwards:

-R

NTFS

ddru_ntfsbitmap will try to provide a map of sectors containing files, so we can focus on rescuing those areas instead of empty or erased areas.

Get it from https://sourceforge.net/p/ddrutility

info ddrutility ddru_ntfsbitmap
sudo ddru_ntfsbitmap -V -i $((2048*512)) wd.img wd.ntfs.map

It first extracts the NTFS boot sector then the MFT to find the position of the NTFS map, then the map.

Strangely, recovering the MFT failed most often but it managed once to proceed and recover the map.

sudo ddru_ntfsbitmap -r -V -i $((2048*512)) /dev/sda wd.ntfs.map
ddru_ntfsbitmap 1.5 20150111

Reading boot sector...
command = ddrescue  -i1048576 -o0 -s512 /dev/sda '__bootsec' '__bootsec.log'
GNU ddrescue 1.21
Press Ctrl-C to interrupt
     ipos:    1048 kB, non-trimmed:        0 B,  current rate:     512 B/s
     opos:        0 B, non-scraped:        0 B,  average rate:     512 B/s
non-tried:        0 B,     errsize:        0 B,      run time:          1s
  rescued:      512 B,      errors:        0,  remaining time:         n/a
percent rescued: 100.00%      time since last successful read:          0s
Finished                                     

Reading bitmap inode from mft...
command = ddrescue  -i3222274048 -o0 -s16384 /dev/sda '__mftshort' '__mftshort.log'
GNU ddrescue 1.21
Press Ctrl-C to interrupt
     ipos:    3222 MB, non-trimmed:        0 B,  current rate:   16384 B/s
     opos:        0 B, non-scraped:        0 B,  average rate:   16384 B/s
non-tried:        0 B,     errsize:        0 B,      run time:          1s
  rescued:    16384 B,      errors:        0,  remaining time:         n/a
percent rescued: 100.00%      time since last successful read:          0s
Finished                                     

............Reading part 0 of $Bitmap...........
command = ddrescue  -i3191742464 -o0 -s30523392 /dev/sda '__bitmapfile' '_part0__bitmapfile.log'
GNU ddrescue 1.21
Press Ctrl-C to interrupt
     ipos:    3222 MB, non-trimmed:        0 B,  current rate:   2482 kB/s
     opos:   30466 kB, non-scraped:        0 B,  average rate:    803 kB/s
non-tried:        0 B,     errsize:        0 B,      run time:         38s
  rescued:   30523 kB,      errors:        0,  remaining time:         n/a
percent rescued: 100.00%      time since last successful read:          0s
Finished                                     

............ Done reading part 0 of $Bitmap...........
Creating ddrescue domain logfile...
end = 1000170586112
total_size = 1000170586112
Finished creating logfile
total= 1000169537536 bytes
used= 375333707776 (37.53%)
free= 624835829760 (62.47%)
ddru_ntfsbitmap took 40.164547 seconds to complete

So we know which blocks contain valid files but we can't get the directory structure.

We can see where those blocks are located:

ddrescueview  wd.ntfs.map

and we can now use it to guide ddrescue:

sudo ddrescue -m wd.ntfs.map /dev/sda wd.img wd.img.map

NTFS documentation

MFT

testdisk

testdisk validates the NTFS boot sectors but fails on MFT:

 MFT and MFT mirror are bad. Failed to repair them.

testdisk on Readynas failed to be installed because it expects libntfs-3g852 but Netgear provide their own ntfs-3g and libntfs-3g851, so:

 laptop# apt-get install equivs
 laptop$ equivs-control libntfs-3g852
 # edit libntfs-3g852 or:
 laptop$ cat << EOF > libntfs-3g852 
 Section: misc
 Priority: optional
 Standards-Version: 3.9.2
 Package: libntfs-3g852
 Description: equivs libntfs-3g852
 long description and info
 .
 second paragraph
 EOF
 laptop$ equivs-build libntfs-3g852
 readynas# dpkg -i /tmp/libntfs-3g852_1.0_all.deb 
 readynas# apt-get install testdisk
 readynas# ( cd /lib/x86_64-linux-gnu/ ; ln -s libntfs-3g.so.851 libntfs-3g.so.852 )
 readynas$ testdisk wd.img

ddru_ntfsfindbad

ddru_ntfsfindbad fails also on MFT:

ddru_ntfsfindbad -V -i $((2048*512)) wd.img wd.img.map 
ddru_ntfsfindbad 1.5 20150109
Reading the logfile into memory...
processed 32136 lines out of 32144 with 0 errors
Reading partition boot sector...
Reading mft inode...
There was an error in reading or processing the main mft record.
Attempting to read the mft mirror...
ERROR! Unable to correctly process the mft from the partition boot sector

manual

From ddru_ntfsbitmap log:

wBytesPerSec = 0x0200
uchSecPerClust = 0x08

n64MFTLogicalClustNum = 0x00000000000C0000
>>> "%x" % (0x8 * 0x200 * 0xC0000 + 512*2048)
'c0100000'

n64MFTMirrLogicalClustNum = 0x0000000000000002
>>> "%x" % (0x8 * 0x200 * 2 + 512*2048)
'102000'

sudo dd if=/dev/sda of=MFTshort bs=512 skip=$((0xc0100000/512)) count=$((0x8 * 0x200 * 4/512))
sudo dd if=/dev/sda of=MFTshortmirr bs=512 skip=$((0x102000/512)) count=$((0x8 * 0x200 * 4/512))

Indeed, MFT and MFTmirror are both corrupted (marked "BAAD") and further attempts to recover or mount the NTFS partition fail because of those markers.

ddrescue views

ddrescueview wd.img.map

ddrescueview  wd.ntfs.map

Reconstructing MFT

GetDataBack Simple in level3 seems to be able to recover files from MFT records.
In our case it found only a directory of pictures.
One can try it for free but saving data requires a license (~70$). Note that with the information about size, sector & cluster, it should be possible to extract the files directly with dd once we know where they are.

file carving

As NTFS filesystem is broken, trying file carving...

fill unused areas

Fill empty space with zeroes to ease photorec:

ddrescue --fill-mode=? /dev/zero wd.img wd.ntfs.map

photorec

It found a variety of files, occasionally guessing the filename from metadata.
To sort recovered files per filetype, from the main recovery directory:

find . -type f |sed 's/.*\.//'|sort|uniq > /dest/ext
for ext in $(cat /dest/ext); do echo $ext; mkdir -p /dest/$ext && mv */*.$ext /dest/$ext/;done
rmdir *
mkdir -p /dest/00windows && mv */* /dest/00windows

Little helper to rename docx files from the first text line: (requires docx2txt)

for i in *.docx ; do
    n=$(docx2txt < "$i" |head -n1|sed 's/^[[:space:]]*//;s/[[:space:]]*$//'|cut -c -50|sed 's/^/ /;s/^ $//;s/[[:space:]][[:space:]]*/ /g')
    echo "$n"
    mv "$i" "${i/.docx}$n.docx"
done

fill mode to find which files are damaged

As per info ddrescue

Proceed till mapfile containing only finished ('+') and bad-sector ('-') blocks.

mkdir mnt
printf "DEADBEEF" > tmpfile
ddrescue --fill-mode=l- tmpfile wd.img wd.img.map
rm tmpfile
mount -o loop,ro wd.img mnt
find mnt -type f -exec grep -l "DEADBEEF" '{}' ';'
umount mnt
# for files with identified bad sectors, try harder, e.g. :
ddrescue -i0x12345000 -s512 -dr9 /dev/sda wd.img wd.img.map
#ddrescue --fill-mode=- /dev/zero wd.img wd.img.map
mount -o loop,ro wd.img mnt

@@ Line 180: / Line 180: @@
 sudo ddrescue -m wd.ntfs.map /dev/sda wd.img wd.img.map
 </pre>
+NTFS documentation
+* http://ftp.kolibrios.org/users/Asper/docs/NTFS/ntfsdoc.html
+* https://technet.microsoft.com/en-us/library/cc781134(v=ws.10).aspx
+* https://blogs.technet.microsoft.com/askcore/2009/12/30/ntfs-metafiles/
+* http://ntfs.com/ntfs-mft.htm
 ==MFT==
 ===testdisk===