Difference between revisions of "Reverse-Engineering"

From YobiWiki
Jump to navigation Jump to search
(Created page with "You'll find a lot of (moderate) reverse-engineering in this wiki but this page aims at providing a list of useful resources. ==Books== * [http://shop.oreilly.com/product/9781...")
 
 
(328 intermediate revisions by 2 users not shown)
Line 1: Line 1:
  +
=Static Analysis Tools=
−
You'll find a lot of (moderate) reverse-engineering in this wiki but this page aims at providing a list of useful resources.
  
  +
==[[IDA Pro]]==
  +
==[https://github.com/google/binnavi Binnavi]==
  +
BinNavi is a binary analysis IDE that allows to inspect, navigate, edit and annotate control flow graphs and call graphs of disassembled code.
  +
<br>It relies on [[IDA Pro]] but [https://github.com/cseagle/fREedom fREedom] project is an attempt to remove that dependency and build on top of [[#Capstone|Capstone]] instead.
   
  +
==[http://metasm.cr0.org/ Metasm]==
  +
Metasm is a cross-architecture assembler, disassembler, compiler, linker and debugger.
  +
<br>It has some advanced features such as live process manipulation, GCC/Microsoft Visual Studio-compatible preprocessor, automatic backtracking in the disassembler (similar to "slicing"), C headers shrinking, linux/windows/remote debugging API interface, a C compiler/decompiler, a gdb-server compatible debugger, and various advanced features. It is written in pure Ruby, with no dependency.
  +
<br>Intel IA32 (16/32/64bits), MIPS, PPC. Ongoing: ARM
  +
<br>MZ, PE/COFF (32 and 64 bits), ELF (32 and 64 bits), Mach-O (incomplete) and UniversalBinary
  +
  +
==[http://www.backerstreet.com/rec/rec.htm REC Studio]==
  +
* x86, x64
  +
* Windows, Linux, Mac OS X
  +
* HLA disassembler
  +
Useful commands:
  +
help
  +
strings
  +
calltree
  +
showprocs
  +
decompile /tmp/myprog.c
  +
click on a function in the "Project" function list to HLA disass it
  +
==[http://www.hopperapp.com/ Hopper]==
  +
* Intel (32 and 64bits), and ARM (ARMv6, ARMv7 and ARM64) processors
  +
* Mach-O binaries (Mac and iOS), PE32/32+/64 Windows binaries and ELF binaries
  +
* decompiler
  +
* debugger
  +
* patcher
  +
==[https://binary.ninja/ Binary Ninja]==
  +
* US$ 99
  +
* Clients: Linux, Windows, Mac OSX
  +
  +
* [https://github.com/Vector35/binaryninja-plugins Plugins]
  +
git clone --recurse https://github.com/Vector35/binaryninja-plugins.git
  +
  +
==[http://www.capstone-engine.org/ Capstone]==
  +
* ARM, ARM64 (ARMv8), Mips, PowerPC, Sparc, SystemZ & Intel
  +
==[https://software.intel.com/en-us/articles/xed-x86-encoder-decoder-software-library Intel XED]==
  +
X86 Encoder Decoder Software Library
  +
<br>Manual [https://software.intel.com/sites/landingpage/xed/ref-manual/html/index.html here]
  +
  +
==[http://www.radare.org/r/ Radare]==
  +
The reverse engineering framework
  +
* [https://github.com/radare/radare2 Git]
  +
* [https://radare.gitbooks.io/radare2book/content/ Book]
  +
* [http://rada.re/get/condret-r2talk.pdf Slides]
  +
* [http://samsymons.com/blog/reverse-engineering-with-radare2-part-1/ tuto]
  +
===[https://inguma.eu/projects/bokken Bokken]===
  +
GUI
  +
<br>[https://github.com/inguma/bokken git repo] synced with mercurial repo
  +
  +
==[https://github.com/bdcht/amoco Amoco]==
  +
Amoco is a python package dedicated to the (static) analysis of binaries
  +
<br>Very young but promising, seems easy to add an arch
  +
<br>With BBL symbolic execution
  +
==[https://code.google.com/p/miasm/ Miasm]==
  +
Miasm is a a free and open source (GPLv2) reverse engineering framework. Miasm aims at analyzing/modifying/generating binary programs. * opening/modifying/generating PE/ELF 32/64 le/be using Elfesteem
  +
* Assembling/Disassembling ia32/ppc/arm
  +
* Representing assembly semantic using intermediate language
  +
* Emulating using jit (dynamic code analysis, unpacking, ...)
  +
* Expression simplification for automatic de-obfuscation
  +
* ...
  +
  +
==[https://github.com/wisk/medusa Medusa]==
  +
Medusa is a disassembler designed to be both modular and interactive. It runs on Windows and Linux
  +
  +
It actually can do more than static
  +
* https://wisk.github.io/how-to-emulate-executable-with-medusa-and-python-part-0.html
  +
  +
==[http://derevenets.com/ Snowman]==
  +
(was called SmartDec)
  +
<br>Native code to C/C++ decompiler
  +
<br>x86 and x86-64 architectures, ELF and PE file formats
  +
<br>IDA Pro & standalone versions, for Windows
  +
<br>Standalone i86 Windows version runs fine under Wine
  +
==[https://zneak.github.io/fcd/ fcd]==
  +
fcd is a LLVM-based native program decompiler.
  +
There is support for custom optimization passes written in Python, helping users defeat custom obfuscation schemes.
  +
  +
==[https://github.com/joelpx/reverse reverse]==
  +
Reverse engineering tool for x86/ARM/MIPS. Generates indented pseudo-C with colored syntax code.
  +
  +
==Misc Static==
  +
===[https://code.google.com/p/distorm/ Distorm]===
  +
diStorm3 is really a decomposer, which means it takes an instruction and returns a binary structure which describes it rather than static text, this is great for advanced binary code analysis
  +
===[http://sourceforge.net/apps/trac/pypeelf PyPEELF]===
  +
PyPEELF is a multi-platform binary editor written in Python, wxPython and BOA Constructor. It allows you to manage binary data in PE32, PE32+ (x64) and ELF binary files.
  +
  +
PyPEELF uses pefile to manage PE32 and PE32+ files and pyelf to manage ELF files. Besides, it uses winappdbg and pydasm in some others features like Task Running Viewer and Disassembling files.
  +
  +
PyPEELF was designed for Reverse Engineers who want to edit or visualize binary file data in multi-platforms. That is why PyPEELF runs under Windows and Unix/BSD operating systems
  +
===[http://decompiler.fit.vutbr.cz/ Retargetable decompiler]===
  +
Support ELF & PE for Intel x86, ARM, ARM+Thumb, MIPS, PIC32, and PowerPC architectures
  +
<br>[http://decompiler.fit.vutbr.cz/decompilation/ Online decompilation service] available!
  +
  +
===[http://binwalk.org/ binwalk]===
  +
Binwalk is a fast, easy to use tool for analyzing and extracting firmware images.
  +
===[http://dax89.comlu.com/?page_id=21 PREF]===
  +
Portable Reverse Engineering Framework
  +
<br>On [https://github.com/Dax89/PREF github]
  +
apt-get install qtbase5-dev ...
  +
qmake
  +
make
  +
===[https://bitbucket.org/mihaila/bindead/wiki/Home Bindead]===
  +
A static analysis tool for binaries.
  +
<br>ELF/PE, x86/x64, IL RREIL, DBI PIN
  +
===[https://hexinator.com/ Hexinator]===
  +
A powerful hexadecimal editor
  +
sudo apt-key adv --keyserver pgp.mit.edu --recv-keys A04A6C4681484CF1
  +
sudo apt-get install apt-transport-https
  +
echo "deb [arch=amd64,i386] https://hexinator.com/downloads/ synalysis non-free" |sudo tee /etc/apt/sources.list.d/hexinator.list
  +
sudo apt-get update
  +
sudo apt-get install hexinator
  +
===[https://github.com/trailofbits/mcsema MC-Semantics]===
  +
A library for translating the semantics of native code to LLVM IR. McSema support translation of x86 machine code, including integer, floating point, and SSE instructions
  +
===[https://github.com/joelpx/reverse joelpx/reverse]===
  +
Reverse engineering (x86 / elf) to pseudo-C
  +
===[http://www.darungrim.org/ DarunGrim]===
  +
A Patch Analysis and Binary Diffing Tool
  +
===[http://www.hiew.ru/ Hiew]===
  +
===[http://www.onlinedisassembler.com/ ODA]===
  +
Online disassembler
  +
  +
==Poor man's tools==
  +
File, -z to uncompress, -s to inspect non-files, e.g. /dev/sda1
  +
file -k [-z] [-s] mybin
  +
Strings
  +
strings [-n min_length] -a -e [s|S|b|l|B|L] mybin
  +
==Android==
  +
===Documentation===
  +
* [http://source.android.com/devices/tech/dalvik/index.html Dalvik]: bytecode, dex & VM instructions
  +
===[http://code.google.com/p/dex2jar/ Dex2jar]===
  +
A tool for converting Android’s .dex format to Java’s .class format
  +
<br>See also [https://code.google.com/p/dex2jar/wiki/DeObfuscateJarWithDexTool DeObfuscate jar with dex tool]
  +
./d2j-dex2jar.sh myapp.apk
  +
This returns a file myapp-dex2jar.jar
  +
<br>Then use Java decompilers: jad, jd-gui, cf below
  +
===[https://github.com/google/enjarify enjarify]===
  +
Similar to dex2jar but newer and supposed to handle cases where dex2jar was failing
  +
===[https://github.com/skylot/jadx Jadx]===
  +
Dex to Java decompiler
  +
  +
===[http://code.google.com/p/smali/ Smali]===
  +
smali/baksmali is an assembler/disassembler for the dex format used by dalvik, Android’s Java VM implementation
  +
  +
Examples:
  +
* https://leonjza.github.io/blog/2015/02/09/no-more-jailbreak-detection-an-adventure-into-android-reversing-and-smali-patching/
  +
  +
===[https://code.google.com/p/android-apktool/ Apktool]===
  +
https://github.com/brutall/brut.apktool
  +
<br>It is a tool for reverse engineering 3rd party, closed, binary Android apps. It can decode resources to nearly original form and rebuild them after making some modifications; it makes possible to debug smali code step by step. Also it makes working with app easier because of project-like files structure and automation of some repetitive tasks like building apk, etc.
  +
apktool d myapp.apk
  +
  +
===[http://apkmultitool.com/node/7 Apk Multi-Tool]===
  +
Swiss knive (was Apk Manager)
  +
<br>Contains apktool, smali/baksmali etc
  +
<br>[https://github.com/APK-Multi-Tool on Github] for Linux release
  +
* 9 decompile apk / 1 select apk / 9 decompile apk
  +
===GetStrings===
  +
Small script to prepare a sed script to inject resource strings into jad, to ease reversing.
  +
<br>''Update:'' inject resource names for other resources than strings, will still be more explanatory than 0x7f123456
  +
<br>To be used e.g. after apktool / Apk Multi-Tool decompilation
  +
<source lang=bash>
  +
#!/bin/bash
  +
DECOMPILED_DIR=working/*apk/
  +
cat $DECOMPILED_DIR/res/values/public.xml|grep "type=.string"|\
  +
sed 's/.*name="\?//;s/" id="\?/ /;s/"\? \/>//'|\
  +
awk --non-decimal-data '{print $2, int($2), $1}'\
  +
> getstring-pub
  +
cat $DECOMPILED_DIR/res/values/strings.xml|grep '<string'|\
  +
sed 's/.*name="\?//; s/"\?>/ /;s/<\/string>//;s/#/\\#/g'\
  +
> getstring-str
  +
join -1 3 -2 1 --nocheck-order getstring-pub getstring-str|\
  +
sed 's/[^ ]\+ \([[:alnum:]]\+\) [[:alnum:]]\+ \(.*\)/s#\1#"\2"#/'\
  +
> getstring-sed
  +
rm getstring-pub getstring-str
  +
cat $DECOMPILED_DIR/res/values/public.xml|grep "type="|\
  +
grep -v "type=.string"|\
  +
sed 's/.*type="\(.*\)" name="\(.*\)" id="\(.*\)" \/>/s#\3#\1:\2#/'\
  +
>> getstring-sed
  +
</source>
  +
  +
===SetStrings===
  +
<source lang=bash>
  +
find $1 -name "*.jad" -exec sed -i -f getstring-sed {} \;
  +
</source>
  +
===[http://www.sable.mcgill.ca/soot/ Soot]===
  +
Soot is a Java bytecode analysis and transformation framework, now supporting Dalvik too.
  +
<br>Get soot.jar
  +
  +
Help:
  +
<source lang=bash>
  +
java -jar soot.jar --help|less
  +
</source>
  +
  +
SootDisassembleApkToJimple.sh
  +
<source lang=bash>
  +
#In case you don't have the right platform android.jar, you can force using another one, e.g.:
  +
#FORCEJAR="-force-android-jar /path/to/android-sdk-linux_x86/platforms/android-17/android.jar"
  +
java -jar soot.jar -allow-phantom-refs -android-jars /path/to/android-sdk-linux_x86/platforms -src-prec apk -process-dir $1 -output-format jimple $FORCEJAR
  +
</source>
  +
  +
SootAssembleJimpleToDex.sh
  +
<source lang=bash>
  +
java -jar soot.jar -allow-phantom-refs -android-jars /path/to/android-sdk-linux_x86/platforms -src-prec jimple -process-dir sootOutput -output-format dex
  +
mv sootOutput/classes.dex .
  +
</source>
  +
  +
===Example===
  +
Example of reverse-engineering and modding APK with smali:
  +
* in '''APK-Multi-Tool-Linux''' working dir:
  +
** Drop myapp.apk in place-apk-here-for-modding/
  +
** ./script.sh (and leave it always open in a separate window)
  +
** 9 decompile / 1 select myapp.apk / 9 decompile
  +
** ./getstrings
  +
** Copy apk to dex2jar working dir
  +
** Copy getstring-sed to jad working dir
  +
* in '''dex2jar''' working dir:
  +
** ./d2j-dex2jar.sh myapp.apk
  +
** Copy myapp-dex2jar.jar to jad working dir (and/or jd-gui)
  +
* in '''jad''' working dir:
  +
** ./unjar myapp-dex2jar.jar
  +
** ./setstrings.sh myapp-dex2jar
  +
** Analyse .jad file and understand what to modify
  +
* in '''jd-gui''' working dir:
  +
** As alternative analysis can also be done with jd-gui directly on .jar file
  +
* in '''APK-Multi-Tool-Linux''' working dir:
  +
** In working/ find corresponding .smali file and modify it
  +
** (in script.sh windows) 13 compile/sign/install
  +
===[http://limelect.com/downloads/apk-shell-decompiler/ Lim Electronics' APK Decompiler]===
  +
A GUI wrapper for apktool.jar, dex2jar.bat and jad.exe on Windows.
  +
  +
===Example 2===
  +
Example of reverse-engineering and modding APK with Soot / jimple
  +
  +
* in '''APK-Multi-Tool-Linux''' working dir:
  +
** Drop myapp.apk in place-apk-here-for-modding/
  +
** ./script.sh (and leave it always open in a separate window)
  +
** 1 extract apk
  +
** Copy apk to soot working dir
  +
* in '''soot''' working dir:
  +
** ./SootDisassembleApkToJimple.sh myapp.apk
  +
** Analyse and modify sootoutput/*.jimple files
  +
** ./SootAssembleJimpleToDex.sh
  +
** Copy classes.dex to overwrite APK-Multi-Tool-Linux/out/classes.dex
  +
* in '''APK-Multi-Tool-Linux''' working dir (in script.sh windows)
  +
** 3 zip apk / 2 regular app
  +
** 4 sign app
  +
** adb install place-apk-here-for-modding/repackaged-signed.apk
  +
  +
===[http://siis.cse.psu.edu/dare/index.html Dare]===
  +
Dalvik Retargeting, a tool for converting Android’s .dex format to Java’s .class format
  +
  +
Retargeted .class:
  +
./dare -d output_dir -e myapp.apk
  +
Optimized retargeted .class: (using Soot, slow!)
  +
./dare -o -d output_dir -e myapp.apk
  +
Decompiled optimized retargeted .class: (using Soot, very slow!)
  +
./dare -c -d output_dir -e myapp.apk
  +
===[https://github.com/honeynet/apkinspector/ APKInspector]===
  +
The goal of this project is to help analysts and reverse engineers to visualize compiled Android packages and their corresponding DEX code. APKInspector provides both analysis functions and graphic features for the users to gain deep insight into the malicious apps
  +
<br>Still beta and inactive for a year.
  +
<br>GUI around other tools
  +
===[http://code.google.com/p/androguard/ Androguard]===
  +
Reverse engineering, Malware analysis of Android applications … and more !
  +
* [https://code.google.com/p/androguard/wiki/Installation Installation]
  +
* [https://code.google.com/p/androguard/wiki/Usage Usage]
  +
* [https://code.google.com/p/androguard/wiki/RE Reverse Engineering Tutorial of Android Apps]
  +
Seems to be able to tackle also dynamically loaded code, native code, reflection code
  +
  +
===[http://code.google.com/p/dex-decomplier/ Dexdump]===
  +
Java .dex file format decompiler
  +
<br>Inactive since 2009
  +
===[http://sseblog.ec-spride.de/tools/flowdroid/ FlowDroid]===
  +
FlowDroid is a context-, flow-, field-, object-sensitive and lifecycle-aware static taint analysis tool for Android applications
  +
===[http://www.mobile-sandbox.com Mobile Sandbox]===
  +
Provides online static analysis of malware images.
  +
===[https://www.pnfsoftware.com/index JEB Decompiler]===
  +
Commercial ($1000)
  +
<br>Decompile Android apps and obfuscated Dalvik bytecode as well as MIPS 32-bit
  +
  +
===[https://github.com/rotlogix/lobotomy Lobotomy]===
  +
Lobotomy is an Android security toolkit that will automate different Android assessments and reverse engineering tasks. The goal of the Lobotomy toolkit is to provide a console environment, which would allow a user to load their target Android APK once, then have all the necessary tools without needing to exit that environment.
  +
<br>Lobotomy leverages and provides wrappers around other popular tools such as Androguard, apktool, Dex2Jar, Frida
  +
<br>Introduction [https://rotlogix.com/2015/12/26/introduction-to-lobotomy-part-one/ part 1], [https://rotlogix.com/2016/01/09/introduction-to-lobotomy-part-two/ part 2], more to come
  +
  +
===[https://github.com/flankerhqd/JAADAS JAADS]===
  +
Joint Advanced Defect Assessment framework for android applications (JAADS, original name JADE) is a tool written in Java and Scala with the power of Soot to provide both interprocedure and intraprocedure static analysis for android applications.
  +
===Misc Android Static===
  +
  +
Online decompilation at http://www.decompileandroid.com/ (using dex2jar, jad, apktool, zip/unzip)
  +
  +
[https://github.com/CalebFenton/simplify Simplify: Generic Android Deobfuscator]
  +
  +
==Java==
  +
===[http://www.varaneckas.com/jad JAD]===
  +
Java Decompiler
  +
<br>To use on a jar (from dex2jar):
  +
<source lang=bash>
  +
#!/bin/bash
  +
JAD=$(pwd)/jad
  +
ODIR=${1%.jar}
  +
if [ "$ODIR" == "$1" ]; then
  +
echo "Error: expecting a file ending with .jar"
  +
exit 1
  +
fi
  +
7z x -o${ODIR} $1
  +
for d in $(find ${ODIR}/com -type d); do
  +
echo Entering $d
  +
cd $d
  +
# Clean Android stuffs
  +
rm *\$*.class
  +
for c in *.class; do
  +
$JAD $c
  +
# Want to keep the .class or not?
  +
rm $c
  +
done
  +
cd -
  +
done
  +
</source>
  +
./unjar myapp-dex2jar.jar
  +
====[http://jadretro.sourceforge.net/ jadretro]====
  +
Helps converting Java 1.4, Java 1.5 or later classes so JAD gives better results
  +
====[http://jad-align.sourceforge.net/ JadAlign]====
  +
Aligns java-files, which are decompiled by jad
  +
java -jar JadHelper-0.0.1.jar myfile.java
  +
No much effect on jad from dex
  +
  +
===[http://jd.benow.ca/ Jd-gui]===
  +
JD-GUI is a standalone graphical utility that displays Java source codes of “.class” files
  +
===[https://github.com/argan/binary-refactor binary-refactor]===
  +
Helper to manual de-obfuscate obfuscated jars
  +
*rename class/packages in a jar
  +
*match a jarjar-ed & obfuscated jar with a known jar,to find the 'same' classes
  +
*bytecode dump(asm)
  +
*class dependency graph
  +
===[http://dirty-joe.com/ dirtyJOE]===
  +
Java Overall Editor is a complex editor and viewer for compiled java binaries (.class files)
  +
===PJB===
  +
* http://blog.soat.fr/tag/jvmhardcore/ (fr)
  +
* http://www.kaourantin.net/source/pbjtools/dpbj.cpp
  +
===Online===
  +
* http://www.javadecompilers.com/
  +
===Procyon===
  +
* https://bitbucket.org/mstrobel/procyon/wiki/Java%20Decompiler
  +
===JEB2===
  +
* https://www.pnfsoftware.com/
  +
===Misc===
  +
* [https://developer.jboss.org/people/ozizka/blog/2014/05/06/java-decompilers-a-sad-situation-of?_sscc=t list]
  +
* [http://www.javadecompilers.com/ Online decompilers] CFR, Jadx, JDCore, Procyon, Fernflower, JAD
  +
  +
==ELF==
  +
man elf
  +
===readelf===
  +
readelf -a -g -t --dyn-syms -W mybin
  +
===elfedit===
  +
===objdump===
  +
objdump -C -g -F -x -T --special-syms mybin
  +
objdump -d -l -r -R -S mybin
  +
objdump -D -l -r -R -S mybin
  +
===nm===
  +
nm -a -C -S -s --special-syms mybin
  +
===ldd===
  +
Shared library dependencies:
  +
ldd -v mybin
  +
  +
==PE==
  +
===[https://code.google.com/p/pefile/ Pefile]===
  +
A Python module to read and work with PE (Portable Executable) files, see [https://code.google.com/p/pefile/wiki/UsageExamples usage examples]
  +
<source lang=python>
  +
#!/usr/bin/env python
  +
import sys, pefile
  +
pe = pefile.PE(sys.argv[1])
  +
pe.dump_info()
  +
open('out.txt', 'w').write(pe.dump_info())
  +
</source>
  +
Can run under Linux
  +
===PEiD===
  +
Can run with Wine
  +
===[http://pe-tools.sourceforge.net/ PETools]===
  +
Can run with Wine
  +
====[http://www.exploit-monday.com/2012/07/get-peheader.html Get-PEHeader]====
  +
A Scriptable In-memory and On-disk PE Parsing Utility
  +
  +
===[http://www.angusj.com/resourcehacker/ Resource Hacker]===
  +
Can run with Wine
  +
===[http://www.dependencywalker.com Dependency Walker]===
  +
Can run with Wine
  +
===[http://wjradburn.com/software/ PEview]===
  +
Can run with Wine
  +
===[http://www.nirsoft.net DLL Export Viewer]===
  +
Can run with Wine
  +
<br>Under Wine, require absolute path to DLL so: click on gears, "load functions from the following DLL file", Browse
  +
===[http://www.smidgeonsoft.prohosting.com/pebrowse-pro-file-viewer.html PEBrowse Pro]===
  +
Can run with Wine
  +
===[http://www.ntcore.com/exsuite.php Explorer Suite]===
  +
* CFF Explorer: Allows also to modify a PE
  +
* Signature Explorer
  +
* PE Detective
  +
* Task Explorer (32 & 64)
  +
  +
===[http://icerbero.com/peinsider/ PE Insider]===
  +
===[http://www.heaventools.com/PE_Explorer_disassembler.htm PE.Explorer]===
  +
  +
===[https://github.com/hfiref0x/WinObjEx64 Windows Object Explorer 64-bit]===
  +
===[https://github.com/mtivadar/qiew Qiew]===
  +
Portable Executable (PE) file viewer
  +
===[http://www.caesum.com/download.php BORG Disassembler ]===
  +
  +
==Static protections==
  +
===Packers===
  +
* http://www.openrce.org/reference_library/packer_database
  +
* http://www.reverse-engineering.info/documents/33.html
  +
* https://corkami.googlecode.com/files/packers.pdf
  +
* [http://upx.sourceforge.net/ UPX]
  +
upx -d myfile
  +
* http://www.woodmann.com/crackz/Packers.htm
  +
* [http://www.crinkler.net/ Crinkler]: some insane PE packing tool coming from the demoscene world.
  +
* [https://github.com/arisada/midgetpack midgetpack] Midgetpack is a binary packer for ELF binaries. The curve25519 is the real advantage of midgetpack. In this mode, you do not provide any password or key. Instead, a key file is generated at packing time. This key file must be used every time you wish to use the binary. When you start the binary, it will give a challenge and expect a response.
  +
* [https://github.com/strazzere/android-unpacker android-unpacker]
  +
  +
==Visualization==
  +
* [https://sites.google.com/site/xxcantorxdustxx/ ..cantor.dust..] ([http://recon.cx/2013/schedule/events/20.html Recon 2013], [https://media.blackhat.com/bh-us-12/Arsenal/Domas/_cantor.dust_.7z.zip BH 2012 demo release]), Windows only?
  +
* [https://github.com/wapiflapi/binglide binglide], Python3
  +
* [http://binwalk.org/ Binwalk] (binwalk --3D), Linux, OSX
  +
* [https://github.com/Spl3en/VisualBinary VisualBinary], Windows, Linux
  +
* [https://code.google.com/p/binvis/ BinVis], C#
  +
* [http://binvis.io/#/ Binvis.io], online
  +
* [https://github.com/letoram/senseye/wiki Senseye], Dynamic Visual Debugging / Reverse Engineering Toolsuite, Linux, à la Cantor Dust
  +
  +
==Source code==
  +
* [http://www.imagix.com/products/source-code-analysis.html Imagix4D]
  +
* [https://scitools.com/ SciTools Understand]
  +
==Firmware==
  +
* [http://binwalk.org/ Binwalk]
  +
* [http://binvis.io Binvis]
  +
* [http://reveng.sourceforge.net/ CRC RevEng]
  +
  +
=Dynamic Analysis Tools=
  +
==[[IDA Pro]]==
  +
  +
==Metasm==
  +
Metasm has debugging capabilities too.
  +
  +
==Intel PIN tools==
  +
* [http://software.intel.com/en-us/articles/pintool Official page]
  +
** [http://software.intel.com/sites/landingpage/pintool/docs/61206/Pin/html/ User guide]
  +
* Windows, Linux, Mac OS X, Android
  +
* x86-32, x86-64 (only Intel platforms obviously)
  +
* binary instrumentation
  +
''The best way to think about Pin is as a "just in time" (JIT) compiler. The input to this compiler is not bytecode, however, but a regular executable. Pin intercepts the execution of the first instruction of the executable and generates ("compiles") new code for the straight line code sequence starting at this instruction. It then transfers control to the generated sequence. The generated code sequence is almost identical to the original one, but Pin ensures that it regains control when a branch exits the sequence. After regaining control, Pin generates more code for the branch target and continues execution. Pin makes this efficient by keeping all of the generated code in memory so it can be reused and directly branching from one sequence to another.''
  +
''In JIT mode, the only code ever executed is the generated code. The original code is only used for reference. When generating code, Pin gives the user an opportunity to inject their own code (instrumentation).''
  +
* [https://software.intel.com/en-us/articles/program-recordreplay-toolkit Program Record/Replay Toolkit]
  +
* [https://github.com/SEDS/PinPP PIN++], see [http://sebox.cs.iupui.edu/PDF/GPCE2014-PinPP.pdf paper (pdf)]
  +
* New attempt to bind with Python: @ancat & @1blankwall1 at Shmoocon2015: [https://raw.githubusercontent.com/blankwall/Python_Pin/master/talk.pdf slides], [https://github.com/blankwall/Python_Pin github]
  +
  +
Tracers:
  +
* [https://bitbucket.org/mihaila/bintrace/wiki/Home BinTrace]
  +
==[http://triton.quarkslab.com/ Triton]==
  +
Triton is a dynamic binary analysis (DBA) framework. It provides internal components like a Dynamic Symbolic Execution (DSE) engine, a Taint Engine, an intermediate representation based on SMT2-Lib of the x86 and x86-64 instructions set, SMT simplification passes, an SMT Solver Interface and, the last but not least, Python bindings.
  +
  +
==[http://www.frida.re/ Frida]==
  +
Inject JavaScript to explore native apps on Windows, Mac, Linux, iOS and Android.
  +
===[https://github.com/dweinstein/awesome-frida Awesome Frida]===
  +
A curated list of awesome projects, libraries, and tools powered by Frida.
  +
===[https://dpnishant.github.io/appmon/ AppMon]===
  +
An automated framework for monitoring and tampering system API calls of native iOS and Android apps
  +
  +
==[http://www.dynamorio.org/ DynamoRIO]==
  +
DynamoRIO is a runtime code manipulation system that supports code transformations on any part of a program, while it executes. DynamoRIO exports an interface for building dynamic tools for a wide variety of uses: program analysis and understanding, profiling, instrumentation, optimization, translation, etc. Unlike many dynamic tool systems, DynamoRIO is not limited to insertion of callouts/trampolines and allows arbitrary modifications to application instructions via a powerful IA-32/AMD64 instruction manipulation library. DynamoRIO provides efficient, transparent, and comprehensive manipulation of unmodified applications running on stock operating systems (Windows or Linux) and commodity IA-32 and AMD64 hardware.
  +
  +
For ARM, see also [https://github.com/sfrankl85/DynamoRIO-ARM DynamoRIO-ARM] and (dead?) [https://github.com/j616/DynamoRIO-for-ARM DynamoRIO-for-ARM]
  +
==[http://bitblaze.cs.berkeley.edu/temu.html TEMU]==
  +
The BitBlaze infrastructure provides a component, called TEMU, for dynamic binary analysis. TEMU is built upon a whole-system emulator, QEMU, and provides the following functionality:
  +
* Dynamic taint analysis. TEMU is able to perform whole-system dynamic taint analysis. Marking certain information sources (e.g., keystrokes, network inputs, reads for certain memory locations, and function call outputs) as tainted, TEMU keeps track of the tainted information propagating in the system. This feature also provides a plug-in environment for dynamic symbolic execution, in which symbolic values are marked as tainted, and concrete values as untainted.
  +
* OS awareness. Information about OS-level abstractions like processes and files is important for many kinds of analysis. Using knowledge of the guest operating system (Windows XP or Linux), TEMU can determine what process and module is currently executing, what API calls have been invoked (with their arguments), and what disk locations belong to which files.
  +
* In-depth behavioral analysis. TEMU is able to understand how an analyzed binary interacts with the environment, such as what API calls are invoked, and what outstanding memory locations are accessed. By marking the inputs as tainted (i.e., symbolic), TEMU provides insights about how outputs are formulated from inputs.
  +
==[http://qira.me/ QIRA]==
  +
QEMU Interactive Runtime Analyser to do dynamic analysis as well as IDA does static analysis
  +
<br>Write-up example: [https://code.google.com/p/qira/wiki/ezhpQIRAwriteup ezhp]
  +
<br>QIRA src now at https://github.com/BinaryAnalysisPlatform
  +
  +
==[http://visi.kenshoto.com/viki/Vdb Vdb/Vtrace] / [http://visi.kenshoto.com/viki/Vivisect Vivisect]==
  +
* debugger, static analysis
  +
* Windows, Linux, Android
  +
* Intel, ARM
  +
''vtrace is a cross-platform process debugging API implemented in python, and vdb is a debugger which uses it''
  +
<br>''vivisect is a Python based static analysis and emulation framework''
  +
* [https://github.com/pdasilva/vtrace_scripts vtrace script examples]
  +
==[http://jolmos.blogspot.com.es/2014/09/s2-dynamic-tracer-and-decompiler-for-gdb.html S2 Dynamic tracer and decompiler for gdb]==
  +
==[https://github.com/frida/cryptoshark Cryptoshark]==
  +
Self-optimizing cross-platform code tracer based on dynamic recompilation, powered by Frida and Capstone
  +
==[http://www.s3.eurecom.fr/tools/avatar/ Avatar]==
  +
Avatar is an event-based arbitration framework that orchestrates the communication between an emulator and a target physical device. Avatar's goal is to enable complex dynamic analysis of embedded firmware in order to assist in a wide range of security-related activities including (but not limited to) reverse engineering, malware analysis, vulnerability discovery, vulnerability assessment, backtrace acquisition and root-cause analysis of known test cases.
  +
<br>ARM
  +
  +
==Android==
  +
===[http://mulliner.org/android/ ADBI: Binary Instrumentation Framework for Android]===
  +
Slides [http://mulliner.org/android/feed/binaryinstrumentationandroid_mulliner_summercon12.pdf here]
  +
* https://github.com/crmulliner/adbi
  +
  +
===[http://mulliner.org/android/ Dynamic Dalvik Instrumentation Framework for Android]===
  +
Slides [http://mulliner.org/android/feed/mulliner_ddi_30c3.pdf here]
  +
* https://github.com/crmulliner/ddi
  +
  +
===[https://code.google.com/p/decaf-platform/ DroidScope]===
  +
DECAF(short for Dynamic Executable Code Analysis Framework) is a binary analysis platform based on QEMU.
  +
This is also the home of the DroidScope dynamic Android malware analysis platform. DroidScope is now an extension to DECAF
  +
<br>Slides [https://www.usenix.org/sites/default/files/conference/protected-files/yan_usenixsecurity12_slides.pdf here] and article [https://www.usenix.org/system/files/conference/usenixsecurity12/sec12-final107.pdf here]
  +
===[https://code.google.com/p/droidbox/ DroidBox]===
  +
Android Application Sandbox
  +
===[http://appanalysis.org/ TaintDroid]===
  +
Realtime Privacy Monitoring on Smartphones
  +
===[http://www.sable.mcgill.ca/soot/ Soot]===
  +
Java, Dalvik (see [http://www.bodden.de/2013/01/08/soot-android-instrumentation/ here] and [http://www.abartel.net/dexpler/ here])
  +
===GameCIH===
  +
===GameGuardian===
  +
===[https://labs.mwrinfosecurity.com/tools/drozer/ Drozer]===
  +
Comprehensive security and attack framework for Android
  +
<br>Interacts with Dalvik VM and explore applications attack surface (activities, content providers, services, etc).
  +
<br>Can also be used remotely à la Metasploit with exploits & payloads
  +
===[https://github.com/swdunlop/AndBug AndBug]===
  +
A Scriptable Debugger for Android's Dalvik Virtual Machine
  +
===[https://github.com/AndroidHooker/hooker Hooker]===
  +
Hooker is an opensource project for dynamic analysis of Android applications. This project provides various tools and applications that can be use to automaticaly intercept and modify any API calls made by a targeted application. It leverages Android Substrate framework to intercept these calls and aggregate all their contextual information (parameters, returned values, ...) in an elasticsearch database. A set of python scripts can be used to automatize the execution of an analysis in order to collect any API calls made by a set of applications.
  +
===[http://repo.xposed.info/module/de.robv.android.xposed.installer Xposed]===
  +
Changes app_process binary and hooks into all system or applications
  +
<br>Many modules
  +
<br>See also [http://forum.xda-developers.com/xposed XDA forum]
  +
===[http://www.cydiasubstrate.com Cydia Substrate]===
  +
Similar to Xposed but not via replacement of system components.
  +
<br>Hooks into Dalvik and native code
  +
===[https://vaioco.github.io/projects/ ARTDroid]===
  +
Simple and easy to use library to intercept virtual-method calls under the Android ART runtime.
  +
* [https://github.com/vaioco/art-hooking-vtable Git]
  +
* [https://vaioco.github.io/art/artdroid-doc/ Documentation]
  +
  +
===Misc Android Dynamic===
  +
* [https://docs.google.com/file/d/0B8LDObFOpzZqY2E1MTIyNzUtYTkzNS00MTUwLWJmODAtZTYzZGY2MDZmOTg1/edit setpropex], as setprop but changes read-only properties by attaching to init via ptrace
  +
* [https://www.isecpartners.com/tools/mobile-security/intent-sniffer.aspx iSec Intent Sniffer] and [https://www.isecpartners.com/tools/mobile-security/intent-fuzzer.aspx iSec Intent Fuzzer]
  +
* [http://www4.comp.polyu.edu.hk/~csxluo/NDroid.pdf On Tracking Information Flows through JNI in Android Applications (pdf)], A dynamic information flow tracing system for Android, sources [https://github.com/0-14N/NDroid here]
  +
* [https://github.com/BaiduSecurityLabs/ZjDroid ZjDroid] Android app dynamic reverse tool based on Xposed framework (you'd better understand Chinese...)
  +
* [https://github.com/robertmillan/mkbreak mkbreak] Generic exploit for master key vulnerability in Android
  +
* [https://manifestsecurity.com/appie/ Appie] Android Pentesting Portable Integrated Environment
  +
* http://translate.wooyun.io/2015/06/17/android-logcat-security.html
  +
  +
==iOS==
  +
===[https://github.com/BishopFox/iSpy iSpy]===
  +
A reverse engineering framework for iOS
  +
  +
==Java==
  +
===[https://code.google.com/p/javasnoop/ Javasnoop]===
  +
A tool that lets you intercept methods, alter data and otherwise hack Java applications running on your computer.
  +
==Flash==
  +
===[https://github.com/ohjeongwook/FlashHacker FlashHacker]===
  +
ActionScript Bytecode instrumentation framework
  +
  +
==ELF==
  +
===ltrace/strace===
  +
Tracing library calls and system calls.
  +
<br>Getting a summary:
  +
ltrace -f -S mybin 2>&1|grep '(.*)'|sed 's/(.*//'|sort|uniq -c
  +
Getting more:
  +
ltrace -f -i -S -n 4 -s 1024 mybin
  +
===[https://github.com/leviathansecurity/ftrace ftrace]===
  +
Tracing inner execution flow as well
  +
  +
===Lib preloading===
  +
<source lang=c>
  +
#define _GNU_SOURCE
  +
  +
#include <dlfcn.h>
  +
#include <sys/types.h>
  +
#include <unistd.h>
  +
#include <errno.h>
  +
#include <stdio.h>
  +
#include <time.h>
  +
  +
  +
// Kill nanosleep()
  +
int nanosleep(const struct timespec *req, struct timespec *rem){
  +
printf("\n==== In our own nanosleep(), I dunnah want sleep\n");
  +
return 0;
  +
}
  +
  +
// Kill usleep()
  +
int usleep(useconds_t usec){
  +
printf("\n==== In our own usleep(), I dunnah want sleep\n");
  +
return 0;
  +
}
  +
  +
// Fix time()
  +
time_t time(time_t *t){
  +
printf("\n==== In our own time(), will return 1380120175\n");
  +
return 1380120175;
  +
}
  +
  +
// Fix srand()
  +
void srand(unsigned int seed){
  +
printf("\n==== In our own srand(), will do srand(0)\n");
  +
void (*original_srand)(unsigned int seed);
  +
original_srand = dlsym(RTLD_NEXT, "srand");
  +
unsigned int myseed = 0;
  +
return (*original_srand)(myseed);
  +
}
  +
  +
#if 0
  +
// Kill rand()
  +
int rand(void){
  +
printf("\n==== In our own rand(), will return 0\n");
  +
return 0;
  +
}
  +
#else
  +
// Intercept rand()
  +
int rand(void){
  +
int (*original_rand)(void);
  +
original_rand = dlsym(RTLD_NEXT, "rand");
  +
int r = (*original_rand)();
  +
printf("\n==== In our own rand(), will return %04X\n", r);
  +
return r;
  +
}
  +
#endif
  +
</source>
  +
gcc -fPIC -shared -Wl,-soname,patch -o patch.so patch.c -ldl
  +
export LD_PRELOAD=patch.so
  +
export LD_LIBRARY_PATH=.:$LD_LIBRARY_PATH
  +
===[https://github.com/poliva/ldpreloadhook ldpreloadhook]===
  +
a quick open/close/ioctl/read/write/free symbol hooker
  +
  +
===[http://stealth.openwall.net/local/ injectso]===
  +
* x86-32, x86-64, ARM (since v0.52)
  +
===[https://code.google.com/p/scanmem/ scanmem]===
  +
scanmem is a simple interactive debugging utility for linux, used to locate the address of a variable in an executing process. This can be used for the analysis or modification of a hostile process on a compromised machine, reverse engineering, or as a "pokefinder" to cheat at video games.
  +
* Linux/Android
  +
* with a GUI since v0.13: GameConqueror
  +
===GDB===
  +
Enable binary writing, here changing a conditional jump to unconditional jump:
  +
gdb -write -silent --args mycode 1 2 3
  +
...
  +
(gdb) set {unsigned char}0x400123 = 0xeb
  +
(gdb) disassemble 0x400123 0x400124
  +
0x400123 jmp 0x...
  +
or injecting NOPs:
  +
(gdb) set {unsigned char}0x400123 = 0x90
  +
====Extensions====
  +
Stephen Bradshaw ha swritten some extensions to have more useful gdb info when debugging stripped binaries, closer to what you get with OllyDbg. See:
  +
* http://www.thegreycorner.com/2013/10/my-python-gdb-extensions.html
  +
* http://www.thegreycorner.com/2014/03/gdb-extensions-110.html
  +
====GUI====
  +
* [https://github.com/snarez/voltron Voltron] is an unobtrusive debugger UI for hackers
  +
* [https://github.com/hexgolems/schem SchemDBG] is a backend agnostic debugger frontend that focuses on debugging binaries without access to the source code
  +
===[http://www.eresi-project.org/ ERESI]===
  +
The ERESI Reverse Engineering Software Interface is a multi-architecture binary analysis framework with a domain-specific language tailored to reverse engineering and program manipulation.
  +
===[http://rr-project.org/ rr]===
  +
rr aspires to be your primary debugging tool, replacing — well, enhancing — gdb. You record a failure once, then debug the recording, deterministically, as many times as you want. The same execution is replayed every time.
  +
<br>rr also provides efficient reverse execution under gdb. Set breakpoints and data watchpoints and quickly reverse-execute to where they were hit.
  +
===[http://angr.io/ angr]===
  +
angr is a framework for analyzing binaries. It focuses on both static and dynamic symbolic ("concolic") analysis, making it applicable to a variety of tasks.
  +
  +
Github: https://github.com/angr
  +
  +
==PE==
  +
===[http://technet.microsoft.com/en-us/sysinternals/bb896645 Process Monitor]===
  +
===[http://technet.microsoft.com/en-us/sysinternals/bb896653 Process Explorer]===
  +
===[http://sourceforge.net/projects/regshot/ RegShot]===
  +
Computes diff between two registry snapshots
  +
===[http://www.nirsoft.net HeapMemView]===
  +
=== OllyDbg ===
  +
  +
PE32-only dynamic disassembler and debugger: http://ollydbg.de/.
  +
<br>Version 1.1 is historically widespread, version 2.0 is re-written from scratch, still considered as beta by some.
  +
<br>Support software and hardware breakpoint, binary patching and repacking, symbol analysis, advanced instruction pattern search, trace with conditional breaking, etc.
  +
=== ImmDbg===
  +
There is also a patched version of OllyDbg with advanced python scripting ability called Immunity Debugger: http://www.immunityinc.com/products-immdbg.shtml
  +
<br>Expect some OllyDbg plugins to not work properly with ImmDbg.
  +
<br>Plugins:
  +
* [http://redmine.corelan.be/projects/mona Mona], a debugger plugin / Exploit Development Swiss Army Knife
  +
  +
===[https://github.com/MarioVilas/winappdbg WinAppDbg]===
  +
The WinAppDbg python module allows developers to quickly code instrumentation scripts in Python under a Windows environment.
  +
====[https://brundlelab.wordpress.com/2012/08/19/small-and-cute-execution-tracer/ Tracer.py]====
  +
Based on WinAppDbg, finds interesting bits in trace by dichotomy signal/noise
  +
* run first time and try everything but not the interesting stuff -> use noise option
  +
* then run again and try interesting stuff -> use signal option
  +
  +
====[https://github.com/carlosgprado/Python-to-the-rescue/blob/master/WTFDLL.py WTFDLL.py]====
  +
Find libraries loaded at runtime and the functions called
  +
  +
===[http://x64dbg.com/#start x64_dbg]===
  +
An open-source x64/x32 debugger for windows.
  +
  +
===[http://www.duxcore.com/index.php/prod/visual-duxdebugger/overview Visual DuxDebugger]===
  +
  +
==[http://www.cuckoosandbox.org/ Cuckoo Sandboxing]==
  +
Currently only supporting Windows binaries.
  +
<br>''Cuckoo Sandbox is a malware analysis system. You can throw any suspicious file at it and in a matter of seconds Cuckoo will provide you back some detailed results outlining what such file did when executed inside an isolated environment.''
  +
''Cuckoo generates a handful of different raw data which include:''
  +
* ''Native functions and Windows API calls traces''
  +
* ''Copies of files created and deleted from the filesystem''
  +
* ''Dump of the memory of the selected process''
  +
* ''Full memory dump of the analysis machine''
  +
* ''Screenshots of the desktop during the execution of the malware analysis''
  +
* ''Network dump generated by the machine used for the analysis''
  +
==[http://www.tetrane.com/ Reven]==
  +
* [http://lifeat.tetrane.com/2014/11/reven-technology-overview.html technology overview]
  +
Unknown price
  +
  +
==Protections==
  +
* http://www.openrce.org/reference_library/anti_reversing
  +
* https://corkami.googlecode.com/files/cm.pdf
  +
* ptrace e.g. [http://www.coredump.gr/articles/ios-anti-debugging-protections-part-1/ on iOS]
  +
* sysctl, e.g. [http://www.coredump.gr/articles/ios-anti-debugging-protections-part-2/ on iOS]
  +
* [http://h4des.org/blog/index.php?/archives/346-ELF-obfuscation-let-analysis-tools-show-wrong-external-symbol-calls.html ELF obfuscation] let analysis tools show wrong external symbol calls
  +
* [https://github.com/obfuscator-llvm/obfuscator/wiki Obfuscator-LLVM]
  +
** [http://blog.quarkslab.com/deobfuscation-recovering-an-ollvm-protected-program.html Deobfuscation: recovering an OLLVM-protected program]
  +
* [http://chaplja.blogspot.in/2015/02/x86obf-code-virtualizer-released-for.html x86obf code virtualizer], 32bit PE files (EXE and DLLs), [http://chaplja.blogspot.in/2015/02/x86obf-source-code.html source code]
  +
  +
=Patching=
  +
* [[IDA_Pro#Patching|IDA Pro DB patching]]
  +
* [[IDA_Pro#Fentanyl|IDA Pro & Fentanyl]]
  +
* [[Reverse-Engineering#Radare|Radare]]
  +
=Fuzzing=
  +
* [http://lcamtuf.coredump.cx/afl/ AFL]
  +
** [https://alexgaynor.net/2015/apr/13/introduction-to-fuzzing-in-python-with-afl/ Introduction to Fuzzing in Python with AFL]
  +
=Z3=
  +
* [http://blog.yurichev.com/node/86 Using Z3 theorem prover to prove equivalence of some bizarre alternative to XOR operation]
  +
* https://github.com/0vercl0k/z3-playground/blob/master/hackingweek-reverse400_z3.py
  +
  +
=Exploitation=
  +
==Tools==
  +
* [http://scoding.de/ropper/ Ropper], rop gadget finder and binary information tool, based on Capstone
  +
* [http://shell-storm.org/project/ROPgadget/ ROPgadget], supports ELF/PE/Mach-O format on x86, x64, ARM, PowerPC, SPARC and MIPS architectures
  +
* [http://ropshell.com/ ROPshell], online, supports ELF/PE/Mach-O format on x86, x64, ARM
  +
* [http://aurelien.wail.ly/nrop/ nROP], semantic ROP search, based on LLVM & Z3
  +
* [https://github.com/pwnies/pwntools pwntools]
  +
* [https://code.google.com/p/peda/ PEDA]: Python Exploit Development Assistance for GDB (x86/x64)
  +
* [https://github.com/hugsy/re-stuff/blob/master/gef.py GEF]: GDB enhanced features - multi-arch (x86/x64/mips/ppc/arm)
  +
* [http://koala.cs.pub.ro/hexcellents/wiki/kb/exploiting/home Hexcellents notes]
  +
* [https://dl.dropboxusercontent.com/u/2595211/ROP_ARMEXP.pdf ROP on ARM (pdf)] by Xipiter / dontstuffbeansupyournose
  +
* Framing Signals a return to portable shellcode: [http://www.ieee-security.org/TC/SP2014/papers/FramingSignals-AReturntoPortableShellcode.pdf article], [https://minemu.org/srop_slides_sp2014.pdf slides]
  +
* [https://github.com/programa-stic/barf-project BARF] : A multiplatform open source Binary Analysis and Reverse engineering Framework
  +
  +
==Mitigation techniques==
  +
Some are taken from excellent Android Hacker's Handbook
  +
===Hardening the Heap===
  +
Hardened version of dlmalloc? Alternatives?
  +
  +
This can be done with LD_PRELOAD, e.g. with [http://goog-perftools.sourceforge.net/doc/tcmalloc.html tcmalloc]
  +
LD_PRELOAD="/usr/lib/libtcmalloc.so"
  +
===Protecting against Integer Overflows===
  +
* Protected calloc?
  +
* Hardened library for safe integer operations: [https://code.google.com/p/safe-iop/ safe_iop]
  +
===Preventing Data Execution===
  +
Set stack (and heap) as non-executable.
  +
<br>Kernel marks stack as executable unless it finds a GNU_STACK program header without executable flag set.
  +
<br>To insert non-exec statement:
  +
flag: -znoexecstack
  +
<br>To test:
  +
/usr/sbin/execstack -q myprog
  +
* "?": myprog has no GNU_STACK -> stack is executable
  +
* "-": stack non-executable
  +
* "X": stack executable
  +
Same:
  +
readelf -a myprog|grep -A1 GNU_STACK
  +
* present? with RW or RWE?
  +
Same:
  +
cat /proc/123/maps|grep -E '(stack|heap)'
  +
* rw or rwx?
  +
To modify existing bin:
  +
/usr/sbin/execstack -s myprog # set executable stack
  +
/usr/sbin/execstack -c myprog # clear
  +
  +
===Max nr of process IDs===
  +
/sbin/sysctl kernel.pid_max
  +
Traditionally 32768
  +
/sbin/sysctl -w kernel.pid_max=4194303
  +
===ptrace===
  +
/sbin/sysctl kernel.yama.ptrace_scope
  +
To allow ptrace:
  +
/sbin/sysctl -w kernel.yama.ptrace_scope=0
  +
===Address Space Layout Randomization===
  +
Bin needs to be compiled position-independent:
  +
CFLAGS: -fPIE
  +
LDFLAGS: -pie
  +
To test:
  +
readelf -h myprog | grep Type:
  +
* DYN? position-independent
  +
* EXEC? Not position-independent
  +
or
  +
readelf -d myprog | grep TEXTREL
  +
Global settings
  +
/sbin/sysctl kernel.randomize_va_space
  +
/sbin/sysctl -w kernel.randomize_va_space=2
  +
* 0 – No randomization. Everything is static.
  +
* 1 – Conservative randomization. Shared libraries, stack, mmap(), VDSO and heap are randomized.
  +
* 2 – Full randomization. In addition to elements listed in the previous point, memory managed through brk() is also randomized.
  +
To disable it locally (in a bash and its children)
  +
setarch `uname -m` -R /bin/bash
  +
On 32 bit systems “ulimit -s unlimited” disables the randomization of the mmap()-ing
  +
  +
===Protecting the Stack===
  +
ProPolice stack protection is enabled by using
  +
flags: -fstack-protector
  +
===Format String Protections===
  +
Enabled by using
  +
flags: -Wformat-security -Werror=format-security
  +
Beware compiler cannot detect all corner cases
  +
<br>See also _FORTIFY_SOURCE=2 for runtime protection against %n
  +
===Read-Only Relocations===
  +
Partial relro enabled by using
  +
flags: -Wl,-z,relro
  +
To test:
  +
readelf -h myprog|grep RELRO
  +
* GNU_RELRO? Partial relro protection present
  +
Full relro enabled by using
  +
flags: -Wl,-z,relro -Wl,-z,now
  +
To test:
  +
readelf -d myprog|grep NOW
  +
* flags NOW? Full relro protection present
  +
===Access Control Mechanisms===
  +
SELinux
  +
===(anti-)anti-debug===
  +
* [http://pferrie.host22.com/papers/antidebug.pdf The "Ultimate" Anti-Debugging Reference]
  +
* [http://mrexodia.cf/reversing/2015/02/05/TitanHide/ TitanHide] Opensource ring0 Windows x64 anti-anti-debug driver
  +
  +
==Static source code analysis and fortifying==
  +
===cppcheck===
  +
cppcheck --quiet --check-config .
  +
cppcheck --xml --xml-version=2 --std=posix --std=c99 \
  +
--enable=style,performance,portability,information,unusedFunction \
  +
-I include --force --inconclusive .
  +
===flawfinder===
  +
flawfinder --quiet --dataonly --singleline --followdotdir .|sort -k 2 -r|less
  +
flawfinder --immediate --dataonly --inputs --followdotdir .
  +
===gcc/clang===
  +
export CFLAGS="-Wall -g -O2 -Wextra -pipe -funsigned-char -fstrict-aliasing -Wchar-subscripts -Wundef -Wshadow -Wcast-align -Wwrite-strings -Wunused -Wuninitialized -Wpointer-arith -Wredundant-decls -Winline -Wformat -Wformat-security -Wswitch-enum -Winit-self -Wmissing-include-dirs -Wmissing-prototypes -Wstrict-prototypes -Wold-style-definition -Wbad-function-cast -Wnested-externs -Wmissing-declarations "
  +
Clang:
  +
export CFLAGS="-Wunreachable-code"
  +
export CFLAGS="$CFLAGS -fno-omit-frame-pointer -D_FORTIFY_SOURCE=2 -fstack-protector"
  +
export LDFLAGS="$LDFLAGS -fsanitizer=address -fno-omit-frame-pointer -D_FORTIFY_SOURCE=2 -fstack-protector"
  +
=Other resources=
 
==Books==
  
==Books==
 
* [http://shop.oreilly.com/product/9781593272890.do The IDA Pro Book, 2nd Edition by Chris Eagle]
  
* [http://shop.oreilly.com/product/9781593272890.do The IDA Pro Book, 2nd Edition by Chris Eagle]
 
* [http://shop.oreilly.com/product/9781597492379.do Reverse Engineering Code with IDA Pro by Dan Kaminsky et al]
  
* [http://shop.oreilly.com/product/9781597492379.do Reverse Engineering Code with IDA Pro by Dan Kaminsky et al]
 
* [http://shop.oreilly.com/product/9781593272906.do Practical Malware Analysis by Michael Sikorski]
  
* [http://shop.oreilly.com/product/9781593272906.do Practical Malware Analysis by Michael Sikorski]
  +
* [http://www.amazon.com/Reversing-Secrets-Engineering-Eldad-Eilam/dp/0764574817 Reversing: Secrets of Reverse Engineering by Eldad Eilam]
−
==Resources==
  
  +
* [http://shop.oreilly.com/product/9781886411791.do Crackproof Your Software by Pavol Cerven]
  +
* [http://www.amazon.com/Surreptitious-Software-Obfuscation-Watermarking-Tamperproofing/dp/0321549252 Surreptitious Software: Obfuscation, Watermarking, and Tamperproofing for Software Protection]
  +
* Wikibooks [https://en.wikibooks.org/wiki/Subject:Software_reverse_engineering Subject:Software_reverse_engineering]
  +
** [https://en.wikibooks.org/wiki/X86_Disassembly x86 Disassembly], [https://en.wikibooks.org/wiki/X86_assembly x86 Assembly]
  +
** [https://en.wikibooks.org/wiki/Reverse_Engineering Reverse Engineering]
  +
* [http://yurichev.com/RE-book.html Reverse Engineering for Beginners], free, by @yurichev
  +
  +
==ELF==
  +
* [http://www.exploit-db.com/wp-content/themes/exploit/docs/33429.pdf Linux x86 Reverse Engineering (pdf)] Shellcode Disassembling and XOR decryption
  +
* [http://www.ouah.org/RevEng/ Introduction to Reverse Engineering Software in Linux]
  +
==ARM==
  +
* [http://simplemachines.it/doc/arm_inst.pdf ARM instruction set (pdf)], slides
  +
==Android==
  +
* Big lists of tools
  +
** http://www.nyxbone.com/malware/android_tools.html
  +
** http://wiki.secmobi.com/tools:android_reversing_analysis
  +
** http://wiki.secmobi.com/tools:android_dynamic_analysis
  +
** https://androidsecuritywiki.com/
  +
** http://www.droidsec.org/wiki/
  +
* [http://techbliss.org/ Techbliss] forum: Android, IDA,...
  +
* [http://blog.simplyadvanced.net/android-how-to-decode-proguards-obfuscated-code-from-stack-trace/ How To Decode ProGuard’s Obfuscated Code From Stack Trace]
  +
* [https://www1.informatik.uni-erlangen.de/filepool/publications/Live_Memory_Forensics_on_Android_with_Volatility.pdf Live Memory Forensics on Android with Volatility (pdf)]
  +
* Dalvik and ART [http://newandroidbook.com/files/Andevcon-DEX.pdf Part 1 (DEX)] & [http://newandroidbook.com/files/Andevcon-ART.pdf Part 2 (ART)], slides
  +
* [http://s4mpl3d.me/?p=162 Reverse engineering & Rebuilding 3rd party, closed, binary Android apps (APK’s) using APKtool], tutorial
  +
  +
==iOS==
  +
* [http://samdmarshall.com/re.html Misc RE resources for OSX & iOS]
  +
* [http://www.slideshare.net/jasonhaddix/pentesting-ios-applications Pentesting iOS applications]
  +
* [http://media.hacking-lab.com/scs3/scs3_pdf/SCS3_2011_Bachmann.pdf iOS applications reverse engineering (pdf)]: slides
  +
==Intel==
  +
* [https://code.google.com/p/corkami/ corkami], [https://code.google.com/p/corkami/wiki/x86oddities?show=content&wl=en x86 oddities] etc
  +
* [https://software.intel.com/sites/landingpage/IntrinsicsGuide/ Intel Intrinsics Guide]
  +
* [http://www.intel.com/content/www/us/en/processors/architectures-software-developer-manuals.html Intel® 64 and IA-32 Architectures Software Developer Manuals]
  +
==List of IR for reverse engineering==
  +
list collected by https://twitter.com/elvanderb
  +
<br>from http://pastebin.com/WzuSRQzi
  +
  +
* LLVM IR:
  +
** [http://zneak.github.io/fcd/2016/02/16/lifting-x86-code.html FCD]
  +
** [https://github.com/trailofbits/mcsema MCSEMA]
  +
** [http://dagger.repzret.org/ Dagger]
  +
* [https://aurelien.wail.ly/nrop/index.html Nrop IR]: mix of QEMU IR ([http://wiki.qemu.org/Documentation/TCG TCG]) and LLVM IR
  +
* [https://github.com/lu-zero/vex/blob/master/pub/libvex_ir.h VEX]
  +
* [http://angr.io/ ANGR] - VEX based
  +
* [https://github.com/BinaryAnalysisPlatform/bap BAP]
  +
* [https://www.zynamics.com/binnavi/manual/html/reil_language.htm REIL]
  +
** https://github.com/Cr4sh/openreil
  +
** https://bitbucket.org/mihaila/bindead/wiki/Home
  +
* [https://github.com/BinaryAnalysisPlatform/bil BIL]
  +
** OCaml
  +
* [http://miasmdoc.ajax.re/ MIASM]
  +
** Python - X86 / ARM / MIPS / SH4 / MSP430
  +
* [https://github.com/jjyg/metasm Metasm]
  +
** Ruby - x86 (16 and 32bits), X86_64, MIPS, PowerPC, Sh4
  +
* [https://api.binary.ninja/binaryninja.LowLevelILExpr.html Binary Ninja IR]:
  +
** C++, Python bindings
  +
* [http://triton.quarkslab.com/ Triton IR]
  +
** C/C++, Python bindings - x86, x86_64
  +
* [https://github.com/bdcht/amoco Amoco]
  +
** Python - arm, msp430, pic, sparc, x64, x86, z80
  +
* [http://www.tetrane.com/en/products.html REVEN]:
  +
** http://doc.tetrane.com/latest/
  +
** "Commercial product, the internal IR is only documented/accessible for some filtered customers though..."
  +
* [http://binsec.gforge.inria.fr/ BinSec IR] - DBA:
  +
* [https://derevenets.com/ SNOWMAN IR]:
  +
** x86-64, arm
  +
** https://github.com/yegord/snowman/blob/master/src/nc/arch/x86/X86InstructionAnalyzer.cpp
  +
** https://github.com/yegord/snowman/blob/master/src/nc/arch/arm/ArmInstructionAnalyzer.cpp
  +
* [https://github.com/wisk/medusa Medusa IR]:
  +
** C++/Python
  +
* r2 IR - [https://github.com/radare/radare2book/blob/master/esil.md ESIL]
  +
  +
==Misc Resources==
 
* [http://reverseengineering.stackexchange.com/ Reverse-Engineering on StackExchange]
  
* [http://reverseengineering.stackexchange.com/ Reverse-Engineering on StackExchange]
  +
** [https://reverseengineering.stackexchange.com/questions/1817/is-there-any-disassembler-to-rival-ida-pro List of disassemblers]
  +
* [http://www.openrce.org/ OpenRCE]
 
* [http://www.hexblog.com/ Hex Blog]
  
* [http://www.hexblog.com/ Hex Blog]
 
* http://www.reverse-engineering.info
  
* http://www.reverse-engineering.info
  +
* [http://brundlelab.files.wordpress.com/2013/09/automating-re-with-python.pdf Automating RE with Python (slides)] by Carlos Prado
−
==Tools==
  
  +
* [https://github.com/REMath/literature_review Literature review], big list of tools
−
===IDA Pro===
  
  +
* [http://pythonarsenal.erpscan.com/ Python Arsenal for Reverse Engineering], big list of tools
−
''IDA Pro combines an interactive, programmable, multi-processor disassembler coupled to a local and remote debugger and augmented by a complete plugin programming environment.''
  
  +
* [http://opensecuritytraining.info/Training.html OpenSecurityTraining], x86, x86-64, ARM, RE, malware RE,... tutorials, slides, videos
−
* [https://www.hex-rays.com/products/ida/index.shtml Official page]
  
  +
* [http://thelegendofrandom.com/blog/sample-page Reverse Engineering Tutorials]
−
* Windows, Linux, Mac OS X
  
  +
* http://deroko.phearless.org/rce.html
−
* x86-32, x86-64, ARM and many others
  
  +
* http://www.woodmann.com/collaborative/tools/index.php/Category:RCE_Tools
−
* ELF, Java bytecode, Dalvik, ARM,...
  
  +
* https://www.peerlyst.com/posts/resource-a-list-of-dynamic-analysis-tools-for-software-susan-parker
−
* disassembler, some debugger
  
−
* extensible through plugins & python (anti-debugger, findcrypt,...)
  
−
** [http://thunkers.net/~deft/code/toolbag/docs.html#Installation IDA toolbag]
  
−
** [https://bitbucket.org/daniel_plohmann/simplifire.idascope/ IDAscope]
  
−
* [https://code.google.com/p/patchdiff2/ patchdiff2]
  
−
* [http://www.zynamics.com/bindiff.html Zynamics bindiff]
  
−
* [http://www.darungrim.org/ DarunGrim], another binary diff tool, opensource but discontinued?
  
   
−
===Hex-Rays===
 +
==Going Maths==
  +
* http://www.reddit.com/r/remath
−
The most important IDA Pro plugin is the [https://www.hex-rays.com/products/decompiler/index.shtml Hex-Rays decompiler]
  
  +
* http://www.reddit.com/r/ReverseEngineering/comments/smf4u/reverser_wanting_to_develop_mathematically/
−
* x86 and ARM
  
  +
* [https://github.com/saswatanand/symexbib A Bibliography of Papers on Symbolic Execution Technique and its Applications]
−
* decompiler
  
  +
* [https://github.com/REMath/literature_review Survey of program analysis research with a focus on machine code]
−
Limitations specific to ARM:
  
  +
−
*floating point instructions are not supported
  
  +
==SDR==
−
*VFP/SIMD/Neon/... instructions are not supported
  
  +
* [http://www.sigidwiki.com/wiki/Signal_Identification_Guide Signal Identification Guide]
−
*functions having an argument that is passed partially on registers and partially on the stack are not supported (e.g. int64 passed in R3 and on the stack)
  
  +
* [http://www.rtl-sdr.com/artemis-free-signal-identification-software/ ARTEMIS] signal identification software (Windows)
−
===Intel PIN tools===
  
−
* [http://software.intel.com/en-us/articles/pintool Official page]
  
−
** [http://software.intel.com/sites/landingpage/pintool/docs/61206/Pin/html/ User guide]
  
−
* Windows, Linux, Mac OS X, Android
  
−
* x86-32, x86-64 (only Intel platforms obviously)
  
−
* binary instrumentation
  
−
''The best way to think about Pin is as a "just in time" (JIT) compiler. The input to this compiler is not bytecode, however, but a regular executable. Pin intercepts the execution of the first instruction of the executable and generates ("compiles") new code for the straight line code sequence starting at this instruction. It then transfers control to the generated sequence. The generated code sequence is almost identical to the original one, but Pin ensures that it regains control when a branch exits the sequence. After regaining control, Pin generates more code for the branch target and continues execution. Pin makes this efficient by keeping all of the generated code in memory so it can be reused and directly branching from one sequence to another.
  
−
In JIT mode, the only code ever executed is the generated code. The original code is only used for reference. When generating code, Pin gives the user an opportunity to inject their own code (instrumentation).''
  

Latest revision as of 19:48, 3 May 2017

Static Analysis Tools

IDA Pro

Binnavi

BinNavi is a binary analysis IDE that allows to inspect, navigate, edit and annotate control flow graphs and call graphs of disassembled code.
It relies on IDA Pro but fREedom project is an attempt to remove that dependency and build on top of Capstone instead.

Metasm

Metasm is a cross-architecture assembler, disassembler, compiler, linker and debugger.
It has some advanced features such as live process manipulation, GCC/Microsoft Visual Studio-compatible preprocessor, automatic backtracking in the disassembler (similar to "slicing"), C headers shrinking, linux/windows/remote debugging API interface, a C compiler/decompiler, a gdb-server compatible debugger, and various advanced features. It is written in pure Ruby, with no dependency.
Intel IA32 (16/32/64bits), MIPS, PPC. Ongoing: ARM
MZ, PE/COFF (32 and 64 bits), ELF (32 and 64 bits), Mach-O (incomplete) and UniversalBinary

REC Studio

  • x86, x64
  • Windows, Linux, Mac OS X
  • HLA disassembler

Useful commands: 

help
strings
calltree
showprocs
decompile /tmp/myprog.c

click on a function in the "Project" function list to HLA disass it

Hopper

  • Intel (32 and 64bits), and ARM (ARMv6, ARMv7 and ARM64) processors
  • Mach-O binaries (Mac and iOS), PE32/32+/64 Windows binaries and ELF binaries
  • decompiler
  • debugger
  • patcher

Binary Ninja

  • US$ 99
  • Clients: Linux, Windows, Mac OSX
git clone --recurse https://github.com/Vector35/binaryninja-plugins.git

Capstone

  • ARM, ARM64 (ARMv8), Mips, PowerPC, Sparc, SystemZ & Intel

Intel XED

X86 Encoder Decoder Software Library
Manual here

Radare

The reverse engineering framework

Bokken

GUI
git repo synced with mercurial repo

Amoco

Amoco is a python package dedicated to the (static) analysis of binaries
Very young but promising, seems easy to add an arch
With BBL symbolic execution

Miasm

Miasm is a a free and open source (GPLv2) reverse engineering framework. Miasm aims at analyzing/modifying/generating binary programs. * opening/modifying/generating PE/ELF 32/64 le/be using Elfesteem

  • Assembling/Disassembling ia32/ppc/arm
  • Representing assembly semantic using intermediate language
  • Emulating using jit (dynamic code analysis, unpacking, ...)
  • Expression simplification for automatic de-obfuscation
  • ...

Medusa

Medusa is a disassembler designed to be both modular and interactive. It runs on Windows and Linux

It actually can do more than static

Snowman

(was called SmartDec)
Native code to C/C++ decompiler
x86 and x86-64 architectures, ELF and PE file formats
IDA Pro & standalone versions, for Windows
Standalone i86 Windows version runs fine under Wine

fcd

fcd is a LLVM-based native program decompiler. There is support for custom optimization passes written in Python, helping users defeat custom obfuscation schemes.

reverse

Reverse engineering tool for x86/ARM/MIPS. Generates indented pseudo-C with colored syntax code.

Misc Static

Distorm

diStorm3 is really a decomposer, which means it takes an instruction and returns a binary structure which describes it rather than static text, this is great for advanced binary code analysis

PyPEELF

PyPEELF is a multi-platform binary editor written in Python, wxPython and BOA Constructor. It allows you to manage binary data in PE32, PE32+ (x64) and ELF binary files.

PyPEELF uses pefile to manage PE32 and PE32+ files and pyelf to manage ELF files. Besides, it uses winappdbg and pydasm in some others features like Task Running Viewer and Disassembling files.

PyPEELF was designed for Reverse Engineers who want to edit or visualize binary file data in multi-platforms. That is why PyPEELF runs under Windows and Unix/BSD operating systems

Retargetable decompiler

Support ELF & PE for Intel x86, ARM, ARM+Thumb, MIPS, PIC32, and PowerPC architectures
Online decompilation service available!

binwalk

Binwalk is a fast, easy to use tool for analyzing and extracting firmware images.

PREF

Portable Reverse Engineering Framework
On github 

apt-get install qtbase5-dev ...
qmake
make

Bindead

A static analysis tool for binaries.
ELF/PE, x86/x64, IL RREIL, DBI PIN

Hexinator

A powerful hexadecimal editor 

sudo apt-key adv --keyserver pgp.mit.edu --recv-keys A04A6C4681484CF1
sudo apt-get install apt-transport-https
echo "deb [arch=amd64,i386] https://hexinator.com/downloads/ synalysis non-free" |sudo tee /etc/apt/sources.list.d/hexinator.list
sudo apt-get update
sudo apt-get install hexinator

MC-Semantics

A library for translating the semantics of native code to LLVM IR. McSema support translation of x86 machine code, including integer, floating point, and SSE instructions

joelpx/reverse

Reverse engineering (x86 / elf) to pseudo-C

DarunGrim

A Patch Analysis and Binary Diffing Tool

Hiew

ODA

Online disassembler

Poor man's tools

File, -z to uncompress, -s to inspect non-files, e.g. /dev/sda1 

file -k [-z] [-s] mybin

Strings 

strings [-n min_length] -a -e [s|S|b|l|B|L] mybin

Android

Documentation

  • Dalvik: bytecode, dex & VM instructions

Dex2jar

A tool for converting Android’s .dex format to Java’s .class format
See also DeObfuscate jar with dex tool 

./d2j-dex2jar.sh myapp.apk

This returns a file myapp-dex2jar.jar
Then use Java decompilers: jad, jd-gui, cf below

enjarify

Similar to dex2jar but newer and supposed to handle cases where dex2jar was failing

Jadx

Dex to Java decompiler

Smali

smali/baksmali is an assembler/disassembler for the dex format used by dalvik, Android’s Java VM implementation

Examples:

Apktool

https://github.com/brutall/brut.apktool
It is a tool for reverse engineering 3rd party, closed, binary Android apps. It can decode resources to nearly original form and rebuild them after making some modifications; it makes possible to debug smali code step by step. Also it makes working with app easier because of project-like files structure and automation of some repetitive tasks like building apk, etc. 

apktool d myapp.apk

Apk Multi-Tool

Swiss knive (was Apk Manager)
Contains apktool, smali/baksmali etc
on Github for Linux release

  • 9 decompile apk / 1 select apk / 9 decompile apk

GetStrings

Small script to prepare a sed script to inject resource strings into jad, to ease reversing.
Update: inject resource names for other resources than strings, will still be more explanatory than 0x7f123456
To be used e.g. after apktool / Apk Multi-Tool decompilation 

#!/bin/bash
DECOMPILED_DIR=working/*apk/
cat $DECOMPILED_DIR/res/values/public.xml|grep "type=.string"|\
    sed 's/.*name="\?//;s/" id="\?/ /;s/"\? \/>//'|\
    awk --non-decimal-data '{print $2, int($2), $1}'\
    > getstring-pub
cat $DECOMPILED_DIR/res/values/strings.xml|grep '<string'|\
    sed 's/.*name="\?//; s/"\?>/ /;s/<\/string>//;s/#/\\#/g'\
    > getstring-str
join -1 3 -2 1 --nocheck-order getstring-pub getstring-str|\
    sed 's/[^ ]\+ \([[:alnum:]]\+\) [[:alnum:]]\+ \(.*\)/s#\1#"\2"#/'\
    > getstring-sed
rm getstring-pub getstring-str
cat $DECOMPILED_DIR/res/values/public.xml|grep "type="|\
    grep -v "type=.string"|\
    sed 's/.*type="\(.*\)" name="\(.*\)" id="\(.*\)" \/>/s#\3#\1:\2#/'\
    >> getstring-sed

SetStrings

find $1 -name "*.jad" -exec sed -i -f getstring-sed {} \;

Soot

Soot is a Java bytecode analysis and transformation framework, now supporting Dalvik too.
Get soot.jar

Help: 

java -jar soot.jar --help|less

SootDisassembleApkToJimple.sh 

#In case you don't have the right platform android.jar, you can force using another one, e.g.:
#FORCEJAR="-force-android-jar /path/to/android-sdk-linux_x86/platforms/android-17/android.jar"
java -jar soot.jar -allow-phantom-refs -android-jars /path/to/android-sdk-linux_x86/platforms -src-prec apk -process-dir $1 -output-format jimple $FORCEJAR

SootAssembleJimpleToDex.sh 

java -jar soot.jar -allow-phantom-refs -android-jars /path/to/android-sdk-linux_x86/platforms -src-prec jimple -process-dir sootOutput -output-format dex
mv sootOutput/classes.dex .

Example

Example of reverse-engineering and modding APK with smali:

  • in APK-Multi-Tool-Linux working dir:
    • Drop myapp.apk in place-apk-here-for-modding/
    • ./script.sh (and leave it always open in a separate window)
    • 9 decompile / 1 select myapp.apk / 9 decompile
    • ./getstrings
    • Copy apk to dex2jar working dir
    • Copy getstring-sed to jad working dir
  • in dex2jar working dir:
    • ./d2j-dex2jar.sh myapp.apk
    • Copy myapp-dex2jar.jar to jad working dir (and/or jd-gui)
  • in jad working dir:
    • ./unjar myapp-dex2jar.jar
    • ./setstrings.sh myapp-dex2jar
    • Analyse .jad file and understand what to modify
  • in jd-gui working dir:
    • As alternative analysis can also be done with jd-gui directly on .jar file
  • in APK-Multi-Tool-Linux working dir:
    • In working/ find corresponding .smali file and modify it
    • (in script.sh windows) 13 compile/sign/install

Lim Electronics' APK Decompiler

A GUI wrapper for apktool.jar, dex2jar.bat and jad.exe on Windows.

Example 2

Example of reverse-engineering and modding APK with Soot / jimple

  • in APK-Multi-Tool-Linux working dir:
    • Drop myapp.apk in place-apk-here-for-modding/
    • ./script.sh (and leave it always open in a separate window)
    • 1 extract apk
    • Copy apk to soot working dir
  • in soot working dir:
    • ./SootDisassembleApkToJimple.sh myapp.apk
    • Analyse and modify sootoutput/*.jimple files
    • ./SootAssembleJimpleToDex.sh
    • Copy classes.dex to overwrite APK-Multi-Tool-Linux/out/classes.dex
  • in APK-Multi-Tool-Linux working dir (in script.sh windows)
    • 3 zip apk / 2 regular app
    • 4 sign app
    • adb install place-apk-here-for-modding/repackaged-signed.apk

Dare

Dalvik Retargeting, a tool for converting Android’s .dex format to Java’s .class format

Retargeted .class: 

./dare -d output_dir -e myapp.apk

Optimized retargeted .class: (using Soot, slow!) 

./dare -o -d output_dir -e myapp.apk

Decompiled optimized retargeted .class: (using Soot, very slow!) 

./dare -c -d output_dir -e myapp.apk

APKInspector

The goal of this project is to help analysts and reverse engineers to visualize compiled Android packages and their corresponding DEX code. APKInspector provides both analysis functions and graphic features for the users to gain deep insight into the malicious apps
Still beta and inactive for a year.
GUI around other tools

Androguard

Reverse engineering, Malware analysis of Android applications … and more !

Seems to be able to tackle also dynamically loaded code, native code, reflection code

Dexdump

Java .dex file format decompiler
Inactive since 2009

FlowDroid

FlowDroid is a context-, flow-, field-, object-sensitive and lifecycle-aware static taint analysis tool for Android applications

Mobile Sandbox

Provides online static analysis of malware images.

JEB Decompiler

Commercial ($1000)
Decompile Android apps and obfuscated Dalvik bytecode as well as MIPS 32-bit

Lobotomy

Lobotomy is an Android security toolkit that will automate different Android assessments and reverse engineering tasks. The goal of the Lobotomy toolkit is to provide a console environment, which would allow a user to load their target Android APK once, then have all the necessary tools without needing to exit that environment.
Lobotomy leverages and provides wrappers around other popular tools such as Androguard, apktool, Dex2Jar, Frida
Introduction part 1, part 2, more to come

JAADS

Joint Advanced Defect Assessment framework for android applications (JAADS, original name JADE) is a tool written in Java and Scala with the power of Soot to provide both interprocedure and intraprocedure static analysis for android applications.

Misc Android Static

Online decompilation at http://www.decompileandroid.com/ (using dex2jar, jad, apktool, zip/unzip)

Simplify: Generic Android Deobfuscator

Java

JAD

Java Decompiler
To use on a jar (from dex2jar): 

#!/bin/bash
JAD=$(pwd)/jad
ODIR=${1%.jar}
if [ "$ODIR" == "$1" ]; then
    echo "Error: expecting a file ending with .jar"
    exit 1
fi
7z x -o${ODIR} $1
for d in $(find ${ODIR}/com -type d); do
  echo Entering $d
  cd $d
  # Clean Android stuffs
  rm *\$*.class
  for c in *.class; do
    $JAD $c
    # Want to keep the .class or not?
    rm $c
  done
  cd -
done

./unjar myapp-dex2jar.jar

jadretro

Helps converting Java 1.4, Java 1.5 or later classes so JAD gives better results

JadAlign

Aligns java-files, which are decompiled by jad 

java -jar JadHelper-0.0.1.jar myfile.java

No much effect on jad from dex

Jd-gui

JD-GUI is a standalone graphical utility that displays Java source codes of “.class” files

binary-refactor

Helper to manual de-obfuscate obfuscated jars

  • rename class/packages in a jar
  • match a jarjar-ed & obfuscated jar with a known jar,to find the 'same' classes
  • bytecode dump(asm)
  • class dependency graph

dirtyJOE

Java Overall Editor is a complex editor and viewer for compiled java binaries (.class files)

PJB

Online

Procyon

JEB2

Misc

ELF

man elf

readelf

readelf -a -g -t --dyn-syms -W mybin

elfedit

objdump

objdump -C -g -F -x -T --special-syms mybin
objdump -d -l -r -R -S mybin
objdump -D -l -r -R -S mybin

nm

nm -a -C -S -s --special-syms mybin

ldd

Shared library dependencies: 

ldd -v mybin

PE

Pefile

A Python module to read and work with PE (Portable Executable) files, see usage examples 

#!/usr/bin/env python
import sys, pefile
pe = pefile.PE(sys.argv[1])
pe.dump_info()
open('out.txt', 'w').write(pe.dump_info())

Can run under Linux

PEiD

Can run with Wine

PETools

Can run with Wine

Get-PEHeader

A Scriptable In-memory and On-disk PE Parsing Utility

Resource Hacker

Can run with Wine

Dependency Walker

Can run with Wine

PEview

Can run with Wine

DLL Export Viewer

Can run with Wine
Under Wine, require absolute path to DLL so: click on gears, "load functions from the following DLL file", Browse

PEBrowse Pro

Can run with Wine

Explorer Suite

  • CFF Explorer: Allows also to modify a PE
  • Signature Explorer
  • PE Detective
  • Task Explorer (32 & 64)

PE Insider

PE.Explorer

Windows Object Explorer 64-bit

Qiew

Portable Executable (PE) file viewer

BORG Disassembler

Static protections

Packers

upx -d myfile
  • http://www.woodmann.com/crackz/Packers.htm
  • Crinkler: some insane PE packing tool coming from the demoscene world.
  • midgetpack Midgetpack is a binary packer for ELF binaries. The curve25519 is the real advantage of midgetpack. In this mode, you do not provide any password or key. Instead, a key file is generated at packing time. This key file must be used every time you wish to use the binary. When you start the binary, it will give a challenge and expect a response.
  • android-unpacker

Visualization

Source code

Firmware

Dynamic Analysis Tools

IDA Pro

Metasm

Metasm has debugging capabilities too.

Intel PIN tools

  • Official page
  • Windows, Linux, Mac OS X, Android
  • x86-32, x86-64 (only Intel platforms obviously)
  • binary instrumentation

The best way to think about Pin is as a "just in time" (JIT) compiler. The input to this compiler is not bytecode, however, but a regular executable. Pin intercepts the execution of the first instruction of the executable and generates ("compiles") new code for the straight line code sequence starting at this instruction. It then transfers control to the generated sequence. The generated code sequence is almost identical to the original one, but Pin ensures that it regains control when a branch exits the sequence. After regaining control, Pin generates more code for the branch target and continues execution. Pin makes this efficient by keeping all of the generated code in memory so it can be reused and directly branching from one sequence to another. In JIT mode, the only code ever executed is the generated code. The original code is only used for reference. When generating code, Pin gives the user an opportunity to inject their own code (instrumentation).

Tracers:

Triton

Triton is a dynamic binary analysis (DBA) framework. It provides internal components like a Dynamic Symbolic Execution (DSE) engine, a Taint Engine, an intermediate representation based on SMT2-Lib of the x86 and x86-64 instructions set, SMT simplification passes, an SMT Solver Interface and, the last but not least, Python bindings.

Frida

Inject JavaScript to explore native apps on Windows, Mac, Linux, iOS and Android.

Awesome Frida

A curated list of awesome projects, libraries, and tools powered by Frida.

AppMon

An automated framework for monitoring and tampering system API calls of native iOS and Android apps

DynamoRIO

DynamoRIO is a runtime code manipulation system that supports code transformations on any part of a program, while it executes. DynamoRIO exports an interface for building dynamic tools for a wide variety of uses: program analysis and understanding, profiling, instrumentation, optimization, translation, etc. Unlike many dynamic tool systems, DynamoRIO is not limited to insertion of callouts/trampolines and allows arbitrary modifications to application instructions via a powerful IA-32/AMD64 instruction manipulation library. DynamoRIO provides efficient, transparent, and comprehensive manipulation of unmodified applications running on stock operating systems (Windows or Linux) and commodity IA-32 and AMD64 hardware.

For ARM, see also DynamoRIO-ARM and (dead?) DynamoRIO-for-ARM

TEMU

The BitBlaze infrastructure provides a component, called TEMU, for dynamic binary analysis. TEMU is built upon a whole-system emulator, QEMU, and provides the following functionality:

  • Dynamic taint analysis. TEMU is able to perform whole-system dynamic taint analysis. Marking certain information sources (e.g., keystrokes, network inputs, reads for certain memory locations, and function call outputs) as tainted, TEMU keeps track of the tainted information propagating in the system. This feature also provides a plug-in environment for dynamic symbolic execution, in which symbolic values are marked as tainted, and concrete values as untainted.
  • OS awareness. Information about OS-level abstractions like processes and files is important for many kinds of analysis. Using knowledge of the guest operating system (Windows XP or Linux), TEMU can determine what process and module is currently executing, what API calls have been invoked (with their arguments), and what disk locations belong to which files.
  • In-depth behavioral analysis. TEMU is able to understand how an analyzed binary interacts with the environment, such as what API calls are invoked, and what outstanding memory locations are accessed. By marking the inputs as tainted (i.e., symbolic), TEMU provides insights about how outputs are formulated from inputs.

QIRA

QEMU Interactive Runtime Analyser to do dynamic analysis as well as IDA does static analysis
Write-up example: ezhp
QIRA src now at https://github.com/BinaryAnalysisPlatform

Vdb/Vtrace / Vivisect

  • debugger, static analysis
  • Windows, Linux, Android
  • Intel, ARM

vtrace is a cross-platform process debugging API implemented in python, and vdb is a debugger which uses it
vivisect is a Python based static analysis and emulation framework

S2 Dynamic tracer and decompiler for gdb

Cryptoshark

Self-optimizing cross-platform code tracer based on dynamic recompilation, powered by Frida and Capstone

Avatar

Avatar is an event-based arbitration framework that orchestrates the communication between an emulator and a target physical device. Avatar's goal is to enable complex dynamic analysis of embedded firmware in order to assist in a wide range of security-related activities including (but not limited to) reverse engineering, malware analysis, vulnerability discovery, vulnerability assessment, backtrace acquisition and root-cause analysis of known test cases.
ARM

Android

ADBI: Binary Instrumentation Framework for Android

Slides here

Dynamic Dalvik Instrumentation Framework for Android

Slides here

DroidScope

DECAF(short for Dynamic Executable Code Analysis Framework) is a binary analysis platform based on QEMU. This is also the home of the DroidScope dynamic Android malware analysis platform. DroidScope is now an extension to DECAF
Slides here and article here

DroidBox

Android Application Sandbox

TaintDroid

Realtime Privacy Monitoring on Smartphones

Soot

Java, Dalvik (see here and here)

GameCIH

GameGuardian

Drozer

Comprehensive security and attack framework for Android
Interacts with Dalvik VM and explore applications attack surface (activities, content providers, services, etc).
Can also be used remotely à la Metasploit with exploits & payloads

AndBug

A Scriptable Debugger for Android's Dalvik Virtual Machine

Hooker

Hooker is an opensource project for dynamic analysis of Android applications. This project provides various tools and applications that can be use to automaticaly intercept and modify any API calls made by a targeted application. It leverages Android Substrate framework to intercept these calls and aggregate all their contextual information (parameters, returned values, ...) in an elasticsearch database. A set of python scripts can be used to automatize the execution of an analysis in order to collect any API calls made by a set of applications.

Xposed

Changes app_process binary and hooks into all system or applications
Many modules
See also XDA forum

Cydia Substrate

Similar to Xposed but not via replacement of system components.
Hooks into Dalvik and native code

ARTDroid

Simple and easy to use library to intercept virtual-method calls under the Android ART runtime.

Misc Android Dynamic

iOS

iSpy

A reverse engineering framework for iOS

Java

Javasnoop

A tool that lets you intercept methods, alter data and otherwise hack Java applications running on your computer.

Flash

FlashHacker

ActionScript Bytecode instrumentation framework

ELF

ltrace/strace

Tracing library calls and system calls.
Getting a summary: 

ltrace -f -S mybin 2>&1|grep '(.*)'|sed 's/(.*//'|sort|uniq -c

Getting more: 

ltrace -f -i -S -n 4 -s 1024 mybin

ftrace

Tracing inner execution flow as well

Lib preloading

#define _GNU_SOURCE

#include <dlfcn.h>
#include <sys/types.h>
#include <unistd.h>
#include <errno.h>
#include <stdio.h>
#include <time.h>


// Kill nanosleep()
int nanosleep(const struct timespec *req, struct timespec *rem){
    printf("\n==== In our own nanosleep(), I dunnah want sleep\n");
    return 0;
}

// Kill usleep()
int usleep(useconds_t usec){
    printf("\n==== In our own usleep(), I dunnah want sleep\n");
    return 0;
}

// Fix time()
time_t time(time_t *t){
    printf("\n==== In our own time(), will return 1380120175\n");
    return 1380120175;
}

// Fix srand()
void srand(unsigned int seed){
    printf("\n==== In our own srand(), will do srand(0)\n");
    void (*original_srand)(unsigned int seed);
    original_srand = dlsym(RTLD_NEXT, "srand");
    unsigned int myseed = 0;
    return (*original_srand)(myseed);
}

#if 0
// Kill rand()
int rand(void){
    printf("\n==== In our own rand(), will return 0\n");
    return 0;
}
#else
// Intercept rand()
int rand(void){
    int (*original_rand)(void);
    original_rand = dlsym(RTLD_NEXT, "rand");
    int r = (*original_rand)();
    printf("\n==== In our own rand(), will return %04X\n", r);
    return r;
}
#endif

gcc -fPIC -shared -Wl,-soname,patch -o patch.so patch.c -ldl
export LD_PRELOAD=patch.so
export LD_LIBRARY_PATH=.:$LD_LIBRARY_PATH

ldpreloadhook

a quick open/close/ioctl/read/write/free symbol hooker

injectso

  • x86-32, x86-64, ARM (since v0.52)

scanmem

scanmem is a simple interactive debugging utility for linux, used to locate the address of a variable in an executing process. This can be used for the analysis or modification of a hostile process on a compromised machine, reverse engineering, or as a "pokefinder" to cheat at video games.

  • Linux/Android
  • with a GUI since v0.13: GameConqueror

GDB

Enable binary writing, here changing a conditional jump to unconditional jump: 

gdb -write -silent --args mycode 1 2 3
...
(gdb) set {unsigned char}0x400123 = 0xeb
(gdb) disassemble 0x400123 0x400124
0x400123 jmp 0x...

or injecting NOPs: 

(gdb) set {unsigned char}0x400123 = 0x90

Extensions

Stephen Bradshaw ha swritten some extensions to have more useful gdb info when debugging stripped binaries, closer to what you get with OllyDbg. See:

GUI

  • Voltron is an unobtrusive debugger UI for hackers
  • SchemDBG is a backend agnostic debugger frontend that focuses on debugging binaries without access to the source code

ERESI

The ERESI Reverse Engineering Software Interface is a multi-architecture binary analysis framework with a domain-specific language tailored to reverse engineering and program manipulation.

rr

rr aspires to be your primary debugging tool, replacing — well, enhancing — gdb. You record a failure once, then debug the recording, deterministically, as many times as you want. The same execution is replayed every time.
rr also provides efficient reverse execution under gdb. Set breakpoints and data watchpoints and quickly reverse-execute to where they were hit.

angr

angr is a framework for analyzing binaries. It focuses on both static and dynamic symbolic ("concolic") analysis, making it applicable to a variety of tasks.

Github: https://github.com/angr

PE

Process Monitor

Process Explorer

RegShot

Computes diff between two registry snapshots

HeapMemView

OllyDbg

PE32-only dynamic disassembler and debugger: http://ollydbg.de/.
Version 1.1 is historically widespread, version 2.0 is re-written from scratch, still considered as beta by some.
Support software and hardware breakpoint, binary patching and repacking, symbol analysis, advanced instruction pattern search, trace with conditional breaking, etc.

ImmDbg

There is also a patched version of OllyDbg with advanced python scripting ability called Immunity Debugger: http://www.immunityinc.com/products-immdbg.shtml
Expect some OllyDbg plugins to not work properly with ImmDbg.
Plugins:

  • Mona, a debugger plugin / Exploit Development Swiss Army Knife

WinAppDbg

The WinAppDbg python module allows developers to quickly code instrumentation scripts in Python under a Windows environment.

Tracer.py

Based on WinAppDbg, finds interesting bits in trace by dichotomy signal/noise

  • run first time and try everything but not the interesting stuff -> use noise option
  • then run again and try interesting stuff -> use signal option

WTFDLL.py

Find libraries loaded at runtime and the functions called

x64_dbg

An open-source x64/x32 debugger for windows.

Visual DuxDebugger

Cuckoo Sandboxing

Currently only supporting Windows binaries.
Cuckoo Sandbox is a malware analysis system. You can throw any suspicious file at it and in a matter of seconds Cuckoo will provide you back some detailed results outlining what such file did when executed inside an isolated environment. Cuckoo generates a handful of different raw data which include:

  • Native functions and Windows API calls traces
  • Copies of files created and deleted from the filesystem
  • Dump of the memory of the selected process
  • Full memory dump of the analysis machine
  • Screenshots of the desktop during the execution of the malware analysis
  • Network dump generated by the machine used for the analysis

Reven

Unknown price

Protections

Patching

Fuzzing

Z3

Exploitation

Tools

  • Ropper, rop gadget finder and binary information tool, based on Capstone
  • ROPgadget, supports ELF/PE/Mach-O format on x86, x64, ARM, PowerPC, SPARC and MIPS architectures
  • ROPshell, online, supports ELF/PE/Mach-O format on x86, x64, ARM
  • nROP, semantic ROP search, based on LLVM & Z3
  • pwntools
  • PEDA: Python Exploit Development Assistance for GDB (x86/x64)
  • GEF: GDB enhanced features - multi-arch (x86/x64/mips/ppc/arm)
  • Hexcellents notes
  • ROP on ARM (pdf) by Xipiter / dontstuffbeansupyournose
  • Framing Signals a return to portable shellcode: article, slides
  • BARF : A multiplatform open source Binary Analysis and Reverse engineering Framework

Mitigation techniques

Some are taken from excellent Android Hacker's Handbook

Hardening the Heap

Hardened version of dlmalloc? Alternatives?

This can be done with LD_PRELOAD, e.g. with tcmalloc 

LD_PRELOAD="/usr/lib/libtcmalloc.so"

Protecting against Integer Overflows

  • Protected calloc?
  • Hardened library for safe integer operations: safe_iop

Preventing Data Execution

Set stack (and heap) as non-executable.
Kernel marks stack as executable unless it finds a GNU_STACK program header without executable flag set.
To insert non-exec statement: 

flag: -znoexecstack


To test: 

/usr/sbin/execstack -q myprog
  • "?": myprog has no GNU_STACK -> stack is executable
  • "-": stack non-executable
  • "X": stack executable

Same: 

readelf -a myprog|grep -A1 GNU_STACK
  • present? with RW or RWE?

Same: 

cat /proc/123/maps|grep -E '(stack|heap)'
  • rw or rwx?

To modify existing bin: 

/usr/sbin/execstack -s myprog # set executable stack
/usr/sbin/execstack -c myprog # clear

Max nr of process IDs

/sbin/sysctl kernel.pid_max

Traditionally 32768 

/sbin/sysctl -w kernel.pid_max=4194303

ptrace

/sbin/sysctl kernel.yama.ptrace_scope

To allow ptrace: 

/sbin/sysctl -w kernel.yama.ptrace_scope=0

Address Space Layout Randomization

Bin needs to be compiled position-independent: 

CFLAGS: -fPIE
LDFLAGS: -pie

To test: 

readelf -h myprog | grep Type:
  • DYN? position-independent
  • EXEC? Not position-independent

or 

readelf -d myprog | grep TEXTREL

Global settings 

/sbin/sysctl kernel.randomize_va_space
/sbin/sysctl -w kernel.randomize_va_space=2
  • 0 – No randomization. Everything is static.
  • 1 – Conservative randomization. Shared libraries, stack, mmap(), VDSO and heap are randomized.
  • 2 – Full randomization. In addition to elements listed in the previous point, memory managed through brk() is also randomized.

To disable it locally (in a bash and its children) 

setarch `uname -m` -R /bin/bash

On 32 bit systems “ulimit -s unlimited” disables the randomization of the mmap()-ing

Protecting the Stack

ProPolice stack protection is enabled by using 

flags: -fstack-protector

Format String Protections

Enabled by using 

flags: -Wformat-security -Werror=format-security

Beware compiler cannot detect all corner cases
See also _FORTIFY_SOURCE=2 for runtime protection against %n

Read-Only Relocations

Partial relro enabled by using 

flags: -Wl,-z,relro

To test: 

readelf -h myprog|grep RELRO
  • GNU_RELRO? Partial relro protection present

Full relro enabled by using 

flags: -Wl,-z,relro -Wl,-z,now

To test: 

readelf -d myprog|grep NOW
  • flags NOW? Full relro protection present

Access Control Mechanisms

SELinux

(anti-)anti-debug

Static source code analysis and fortifying

cppcheck

cppcheck --quiet --check-config .
cppcheck --xml --xml-version=2 --std=posix --std=c99 \
  --enable=style,performance,portability,information,unusedFunction \
  -I include --force --inconclusive .

flawfinder

flawfinder --quiet --dataonly --singleline --followdotdir .|sort -k 2 -r|less
flawfinder --immediate --dataonly --inputs --followdotdir .

gcc/clang

export CFLAGS="-Wall -g -O2 -Wextra -pipe -funsigned-char -fstrict-aliasing -Wchar-subscripts -Wundef -Wshadow -Wcast-align -Wwrite-strings -Wunused -Wuninitialized -Wpointer-arith -Wredundant-decls -Winline -Wformat -Wformat-security -Wswitch-enum -Winit-self -Wmissing-include-dirs -Wmissing-prototypes -Wstrict-prototypes -Wold-style-definition -Wbad-function-cast -Wnested-externs -Wmissing-declarations "

Clang: 

export CFLAGS="-Wunreachable-code"
export CFLAGS="$CFLAGS -fno-omit-frame-pointer -D_FORTIFY_SOURCE=2 -fstack-protector"
export LDFLAGS="$LDFLAGS -fsanitizer=address -fno-omit-frame-pointer -D_FORTIFY_SOURCE=2 -fstack-protector"

Other resources

Books

ELF

ARM

Android

iOS

Intel

List of IR for reverse engineering

list collected by https://twitter.com/elvanderb
from http://pastebin.com/WzuSRQzi

Misc Resources

Going Maths

SDR

Retrieved from "https://wiki.yobi.be/index.php?title=Reverse-Engineering&oldid=10289"