Difference between revisions of "Android"

From YobiWiki
Jump to navigation Jump to search
Line 432: Line 432:
 
RsyncArgs: add = --one-file-system
  
RsyncArgs: add = --one-file-system
 
Optional: Any excludes for files you don’t need to back up.
  
Optional: Any excludes for files you don’t need to back up.
  +
Note that in the mentioned link he's using RsyncShareName = / and playing with BackupFilesOnly but for me it looks like BackupFilesOnly was not respected, so I preferred to have separate RsyncShareName and enforcing the --one-file-system option
⚫
I had errors "Ping too slow" so I increased
  
  +
<br>Mmm obviously it won't backup the various /mnt/asec/ mounted containers but the original encrypted container should be in one of the other partitions, no?? I've to understand better that APP2SD stuff...
  ⚫
<br>I had errors "Ping too slow" so I increased
 
PingMaxMsec = 400
  
PingMaxMsec = 400
 
as anyway it's on local network
  
as anyway it's on local network

Revision as of 21:56, 10 November 2011

Links

App stores

Alternate views on the official market:

Alternate markets:

User manuals

Some internals info here

Nexus S

Versions

physical sticker behind battery

  • Model: GT-I9023
  • FCC ID: A3LGTI9023
  • SSN: -I9023GSMH
  • IMEI: xxxxxxx
  • S/N: xxxxxxx

under fastboot, stock

  • Product name - HERRING
  • HW Version - rev 52
  • Bootloader version - I9020XXKA3
  • Baseband version - I9020XXKB3
  • Carrier info - EUR
  • Serial number - xxxxxxx

under fastboot, after upgrade to 2.3.4

  • Baseband version - I9020XXKD1
  • Carrier info - EUR

under 'About phone' from the settings, stock 2.3.3

  • Android 2.3.3
  • Baseband I9023XXKB3
  • Kernel 2.6.35.7-g1d030a7
  • Build GRI54

under 'About phone' from the settings, after upgrade to 2.3.4

  • Android 2.3.4
  • Baseband I9023XXKD1
  • Kernel 2.6.35.7-ge382d80 android-build@apa28 #1
  • Build GRJ22

USB permissions on the host

On the host, example how to solve permissions:
Create /etc/udev/rules.d/99-android.rules for Nexus phones: 

SUBSYSTEMS=="usb", ATTRS{idVendor}=="0bb4", ATTRS{idProduct}=="0fff", MODE="0666", OWNER="<your_account>" # Nexus One Fastboot
SUBSYSTEMS=="usb", ATTRS{idVendor}=="18d1", ATTRS{idProduct}=="4e11", MODE="0666", OWNER="<your_account>" # Nexus One Normal
SUBSYSTEMS=="usb", ATTRS{idVendor}=="18d1", ATTRS{idProduct}=="4e12", MODE="0666", OWNER="<your_account>" # Nexus One Debug/Recovery
SUBSYSTEMS=="usb", ATTRS{idVendor}=="18d1", ATTRS{idProduct}=="4e20", MODE="0666", OWNER="<your_account>" # Nexus S Fastboot
SUBSYSTEMS=="usb", ATTRS{idVendor}=="18d1", ATTRS{idProduct}=="4e21", MODE="0666", OWNER="<your_account>" # Nexus S Normal
SUBSYSTEMS=="usb", ATTRS{idVendor}=="18d1", ATTRS{idProduct}=="4e22", MODE="0666", OWNER="<your_account>" # Nexus S Debug/Recovery
SUBSYSTEMS=="usb", ATTRS{idVendor}=="18d1", ATTRS{idProduct}=="4e23", MODE="0666", OWNER="<your_account>" # ???
SUBSYSTEMS=="usb", ATTRS{idVendor}=="18d1", ATTRS{idProduct}=="4e24", MODE="0666", OWNER="<your_account>" # Nexus S USB tethering
SUBSYSTEMS=="usb", ATTRS{idVendor}=="0bb4", ATTRS{idProduct}=="0c8b", MODE="0666", OWNER="<your_account>" # HTC Normal

Then execute /etc/init.d/udev reload

Restoring factory ROM (2.3.3)

Warning, it will destroy everything, make your backups first!!

  • Get firmware here for a European Nexus S i9023, mine needs the I9023XXKB3 one.
  • Rename tar.md5 as tar
  • Get Odin sw from here, choose i9023
    • e.g. I9003_Odin3_v1.82.rar & SAMSUNG_USB.rar
  • Run Odin (works in a virtualbox if needed), load the 4 files:
    • bootloader: I9023_EUR_GRI54_XXKB3/I9023_EUR_GRI54_XXKB3/Bootloader_I9023XXKA3.tar
    • PDA: I9023_EUR_GRI54_XXKB3/I9023_EUR_GRI54_XXKB3/PDA_SOJU_GRI54_TMO_EUR_MR1_SIGNED.tar
    • Phone: I9023_EUR_GRI54_XXKB3/I9023_EUR_GRI54_XXKB3/MODEM_I9023XXKB3_REV_00_CL912571_SIGNED.tar
    • CSC: I9023_EUR_GRI54_XXKB3/I9023_EUR_GRI54_XXKB3/DGS_I9023_EUR.tar
  • Turn phone off
  • Plug USB
  • Just before battery icon showing, enter download mode by pressing middle of volume up / volume down button for long, you'll get a big yellow warning triangle
  • If using virtualbox, bring USB device (Samsung serial) to virtualbox
  • Now Odin should show a yellow rectangle with COM0
  • Press "start" in Odin

Sources:

Restoring (most of) factory ROM (2.3.3) with fastboot

Ok previous section was about restoring *everything* as genuine but it requires Windows and most of the time all you screwed up was the boot image, the recovery image or the system image so here is how to restore them or part of them provided that you can still enter fastboot: 

fastboot oem unlock                  # if needed, WARNING IT DELETES EVERYTHING!!!!!
fastboot flash recovery recovery.img # from Samsung PDA_SOJU_GRI54_TMO_EUR_MR1_SIGNED.tar
fastboot flash system system.img     # from Samsung PDA_SOJU_GRI54_TMO_EUR_MR1_SIGNED.tar
fastboot flash boot boot.img         # from Samsung PDA_SOJU_GRI54_TMO_EUR_MR1_SIGNED.tar

Warning: I got /sdcard content erased, maybe because I tried to enter the native recovery, but anyway, make backups first before trying anything you risk regretting!!!

E.g. to make a backup with clockwordmod:

  • Put phone into fastboot mode
  • Enter recovery
  • Choose backup & restore / backup
  • Choose mount / mount USB storage
  • From host: copy clockwordmod/backup/*

Entering fastboot mode

  • Power off phone
  • Press Volume up and power button together for long
  • You've entered fastboot
  • Alternatively, fastboot can be triggererd from adb: adb reboot-bootloader

For getting fastboot on the host, see further below

Rooting 2.3.3

!! Don't touch boot image or try CW 3.0.0.0, one of them caused the phone to not start android anymore, I had to perform the full factory restore with Odin !!

  • enter recovery mode from fastboot
    • You can make a backup now
    • install zip from sdcard -> choose -> su-2.3.6.1-ef-signed.zip
    • reboot

Sources:

Note that apparently there is a technique avoiding the full wiping, described here. Not tested.

Upgrading to 2.3.4

A new version was announced (fr) two days ago.
Official way is apparently to type "*#*#2432546#*#*" while using Wi-Fi but all I got was a "checkin succeeded" notification. Anyway patched won't apply cleanly on my rooted phone so better to do it manually.
For GRI54, update.zip is available here.
I tried to apply it but there are several caveats given the previous hacks:

  • boot.img: to be able to patch it I've to restore the original boot.img, loosing temporarily the ro.secure=0 setting (cf "adb as root" paragraph). And failing to patch it would probably mean non-bootable as we would have missed replacing the kernel!
  • radio.img: hash checksum failed, it seems to indicate that expected radio.img is not the one I have.
  • recovery.img: we want to keep the clockworkmod one, so we just skip it for now

To apply those change this means:

  • fastboot flash boot boot.img (from PDA_SOJU_GRI54_TMO_EUR_MR1_SIGNED.tar)
  • edit update.zip to remove radio.img, recovery/ and edit META-INF/com/google/android/updater-script
    • remove all commands about radio & recovery patch
    • add following line to keep rooted: set_perm(0, 0, 6755, "/system/bin/su");
    • if that line is not added, patched phone will not be rooted anymore, which can be easily fixed by applying the su..zip again
  • upload update.zip to /sdcard and apply zip via clockworkmod recovery, it'll skip signature verification by default

Ok now we got a system & boot images upgraded to 2.3.4
We can again modify boot.img to restore ro.secure=0:

  • Extract patched 2.3.4 boot.img (cf below, or use clockworkmod), modify it & flash it back

I also wanted to patch the stock recovery image, just to get a 2.3.4 stock recovery in case of.

  • Install the 2 files from update.zip#recovery/ into /system and chmod 755 /system/etc/install-recovery-sh
  • Restore the stock 2.3.3 recovery.img from PDA_SOJU_GRI54_TMO_EUR_MR1_SIGNED.tar: adb push recovery.img /sdcard/
  • If you've flash_image on the phone you can try: flash_image recovery /sdcard/recovery.img
  • Else:
fastboot erase recovery
fastboot flash recovery recovery.img
  • Reboot the phone & start Android
  • It should be done, recovery should have been patched, you can remove the 2 files we've put in /system and extract the patched recovery image manually (cp /dev/mtd/mtd3 /sdcard/recovery.img). Hash changed so I assume it's properly patched
  • Restore clockworkmod recovery image

What's new?
I didn't mention it but with 2.3.3 I had two problems I was still busy trying to solve: GPS never fixing, zero satellite! And no way to see I get copy-protected applications (and that's not because of rooting the phone, on a stock phone it failed too).
Now GPS fixed quite fast with 2.3.4 and I could for the first time see & download copy-protected apps \o/ (even when the phone was rooted again).

UPDATE
here is the full ROM update, apparently with a new radio which should be ok for all phones...
Seen in this thread.

Upgrading to 2.3.6

I saw there was also newer versions of ClockworkMod, probably better than the preview release I was still using.
Latest ClockworkMod recoveries are here.
For Nexus S, look for "crespo" img

  • Go to fastboot (vol-up + power)
  • Go to recovery
  • Backup & restore / Backup
  • Mount USB
  • Copy all /sdcard content to PC
  • Reboot -> enter fastboot again
  • On PC: fastboot flash recovery recovery-clockwork-5.0.2.0-crespo.img

For GRJ22, upgrade.zip is available here

  • fastboot flash boot boot.img (from stock GRJ22)
  • edit update.zip to remove recovery/ and edit META-INF/com/google/android/updater-script
    • remove all commands about recovery
    • add following line to keep rooted: set_perm(0, 0, 6755, "/system/bin/su");
    • because I've mangled cacerts.bks to add CACert stuff, I had also to remove stuffs about cacerts.bks in the script, then make a separate zip where I first restore the previous stock cacerts.bks from GRJ22 then apply the patch. And finally mangle the new cacerts.bks again.
  • In updater-script, check also which radio version it's able to patch. Look for line similar to this one:
apply_patch("MTD:radio:12583040:2ea138c96cc213b2662a4ae1ddee2d5c6bbcc958:12583040:213c2022516ba651f62064e4379487af1e8499a2",
           "-", 213c2022516ba651f62064e4379487af1e8499a2, 12583040,
           2ea138c96cc213b2662a4ae1ddee2d5c6bbcc958, package_extract_file("radio.img.p"));

Here it expects a radio.img from GRJ22 with SHA1 = 2ea138c96cc213b2662a4ae1ddee2d5c6bbcc958, ok
In case you don't have the right radio img in place, or don want to patch it, remove radio.img from the zip and all commands about radio in updater-script.

  • upload update.zip to /sdcard and "apply update from sdcard" via clockworkmod recovery, it'll skip signature verification by default
  • Backup & Restore / Backup
  • Mount USB
  • Copy new backup to PC

Ok now we got a system & boot images upgraded to 2.3.4
We can again modify boot.img to restore ro.secure=0:

  • Extract patched 2.3.6 boot.img (cf below, or use clockworkmod), modify it & flash it back

Extracting manually images from phone

On root shell on the phone: 

# cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00200000 00040000 "bootloader"
mtd1: 00140000 00040000 "misc"
mtd2: 00800000 00040000 "boot"
mtd3: 00800000 00040000 "recovery"
mtd4: 1d580000 00040000 "cache"
mtd5: 00d80000 00040000 "radio"
mtd6: 006c0000 00040000 "efs"
# cat /dev/mtd/mtd5 > /sdcard/radio.img

etc

Battery

Interesting links:

Android 2.3

Getting fastboot & Android sources

There are several binaries around but I wanted to build my own. Maybe not the shortest way... I downloaded all android sources...
cf http://source.android.com/source/downloading.html
Some missing deps on my 64-but debian when I tried to compile everything: gperf, libc6-dev-i386, lib32ncurses5-dev ia32-libs g++-multilib lib32z1-dev lib32readline6-dev 

. build/envsetup.sh
lunch crespo-eng
make -j2

Maybe we can just compile adb & fastboot: 

make adb
make fastboot

cf http://www.cduce.org/~abate/build-android-adb-debian-sid-amd64

Getting adb & Android SDK

Get it from http://developer.android.com/sdk/index.html
Run tools/android -> in installed packages make sure to have "Android SDK Tools", latest revision and "Android SDK Platform-tools", latest revision
It brings also adb but not fastboot
If you want to update SDK: tools/android update sdk

adb

You need to activate USB debugging:

  • Settings > Applications > Development > USB debugging

Some examples:

  • adb devices
  • adb shell
  • adb logcat

adb as root

Once the phone is rooted, you can, from a shell with e.g. adb shell invoke "su" & get root.
But to get immediately into root, the file /default.prop needs to contain ro.secure=0
But that file is restored from boot.img at each boot so you need to modify that one
See http://android-dls.com/wiki/index.php?title=HOWTO:_Unpack%2C_Edit%2C_and_Re-Pack_Boot_Images#Alternative_Method
but there is also abootimg in debian:

  • Taking the boot.img from Samsung (see above in I9023_EUR_GRI54_XXKB3/I9023_EUR_GRI54_XXKB3/PDA_SOJU_GRI54_TMO_EUR_MR1_SIGNED.tar).
abootimg -x boot.img
mkdir ramdisk
cd ramdisk
gzip -dc ../initrd.img | cpio -i
sed -i default.prop 's/ro.secure=1/ro.secure=0/'
find . -print|cpio -o -Hnewc|gzip > ../initrd.img2
cd ..
abootimg -u boot.img -r initrd.img2
fastboot flash boot boot.img

For the last command, the phone needs of course to be in fastboot mode
Reboot phone
I had to re-enable USB debugging but now adb shell brings me immediately a root shell :)
Note that link mentioned above proposes an alternative way to flash the boot partition, directly from adb shell as root

Without this setup, it's a bit cumbersome to automate root commands from host, it looks like 

adb shell su -c "netcfg usb0 dhcp"

and the SuperUser app prompts you for confirmation on the phone for each new command

Got a failure when trying to flash back a 8Mb boot.img?
From example above I started from a boot.img smaller than the full boot partition but if you create a new boot.img or start from an image of the full partition taken manually or with clockworkmod, boot.img will be 8Mb-large (8388608) and fastboot fails with "FAILED exceed blocks 0x00000020 > 0x0000001e".
I'm not really sure about what size the file should be but as it's filled with zeroes till reaching 8Mb, I decided to cut it:
0x00000020 => 0x0000001e means for me 8388608 / 0x20 * 0x1e = 7864320, so I did: 

dd if=boot.img of=boot2.img bs=262144 count=30 
fastboot flash boot boot2.img

And it worked!

Images structure

You may want to explore .img content (from backups, stock ROMs etc).
Actually all .img are not the same

bootloader.img

/dev/mtd/mtd0
Unknown format

radio.img

/dev/mtd/mtd5
Unknown format, for baseband

recovery.img

/dev/mtd/mtd3
Unknown format

boot.img

/dev/mtd/mtd2
See here for details on the structure, and abootimg on Debian

system.img

Yaffs2 image, can be unpacked with unyaffs
Note that unyaffs failed unpacking stock system.img 2.3.3 & userdata.img but works fine on clockworkmod backups

data.img

Yaffs2 image, can be unpacked with unyaffs

cache.img

/dev/mtd/mtd4
Yaffs2 image, can be unpacked with unyaffs

misc

/dev/mtd/mtd1
Not backuped by clockworkmod

efs

/dev/mtd/mtd6
Yaffs2 image, can be unpacked with unyaffs
Not backuped by clockworkmod
Contains stuffs linked to baseband & bluetooth

Screenshots

Run ddms (from SDK) -> Tools / Device / Screen capture

USB tethering

Plug phone & PC via USB
Activate USB tethering (Settings / Wireless & networks / Tethering / USB Tethering)
It works OOB on Debian, nothing to do

Getting busybox

Need rooted phone, see above
Google's stripped busybox, called toolbox, is far from enough once you get a shell on the phone

Examples to use busybox versions instead of toolbox versions when the command exists twice: 

# busybox mount -o remount,rw /system
# /system/xbin/mount -o remount,rw /system

Wi-Fi & client certs

To be able to authenticate to a Wi-Fi network using client certificates via TLS:
If needed, export certificate from IE in Pkcs#12 PFX, *with* private key, *with* all certs, *without* strong enc, *without* deletion of private key.
Rename .pfx file as .p12
(source: http://www.google.com/support/mobile/bin/answer.py?answer=168466&topic=27214#1086573)
Copy pkcs#12 certificate to root of USB storage.
File must end with .p12, not .pfx!
One single file with key+cert+cacerts is ok
Wi-Fi params: 802.1x EAP / TLS / phase2: None / CA cert: cf previous import / user cert: idem / Identity: DOMAIN\user... / Anonymous id: empty / password:...

Note that after each reboot, you'll have to select manually one of the protected networks to unlock the secure storage of personal certificates or open manually the certificates storage:
Settings > Location & Security > Use secure credentials
See also Keystore Unlocker

Importing certs

I could only manage it via a webserver & crafted headers: 

<?php
header("Content-Type: application/x-x509-ca-cert");
?>
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----

You may try this free service: http://www.realmb.com/droidCert/ which seems to do the same.
But even if imported they seem not to be used e.g. for IMAP TLS.

Importing CA certs in /system

Source: CACert wiki.
You don't need the full Android SDK, just adb binary.
I'm not sure if it's really the proper way but to get recognized the BouncyCastle lib which was already on my system (apt-get install libbcprov-java) I did 

sudo ln -s /usr/share/java/bcprov.jar /usr/lib/jvm/java-6-sun/jre/lib/ext/


Adding CACert certificates: 

adb pull /system/etc/security/cacerts.bks
wget http://www.cacert.org/certs/root.crt
wget http://www.cacert.org/certs/class3.crt
keytool -keystore cacerts.bks -storetype BKS -provider org.bouncycastle.jce.provider.BouncyCastleProvider -storepass changeit -importcert -trustcacerts -alias CACERT -file root.crt
keytool -keystore cacerts.bks -storetype BKS -provider org.bouncycastle.jce.provider.BouncyCastleProvider -storepass changeit -importcert -trustcacerts -alias CACERT3 -file class3.crt 
adb shell busybox mount -o remount,rw /system
adb push cacerts.bks /system/etc/security
adb shell busybox mount -o remount,ro /system

Now my IMAP TLS which is using a CACert-signed certificate works with strict TLS setting on the phone, cool!

WARNING this has broken proper upgrades and I had to mangle the update.zip to restore first the original cacert.bks file then get it patched.
Before I patched it again, my imap server gave me a lot of "couriertls: read: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number" errors before I realized it was because I didn't' have the CACert certificates anymore...

Applications

See Android Apps

Applications development

See Android SDK

Backuping via BackupPC

I'm a big fan of BackupPc and some people managed to do it so I gave it a try.
Check the mentioned link but his setup is a bit different, running CyanogenMod while I'm using a stock fw.
Instructions here suppose your phone is rooted.

IP

Backuppc server needs to reach the phone so your phone needs a static (or DHCP statically attributed) IP or whatever dyndns system.

SSH

I'm using SshDroidPro
Make sure backuppc key is properly installed in /data/data/berserker.android.apps.sshdroidpro/home/.ssh/authorized_keys
Then test it as user backuppc, trying to access the phone and accept the server key fingerprint.

rsync

To get rsync binary, I found rsync backup for Android which downloads a rsync binary during install (a weird way to deal with a GPL program IMHO).
The actual binary it downloads is available here.
But Android wget doesn't support https so you've to transfer it to your phone by another mean.
One way is to install the application I mentioned and let it download that binary.
Then, to install it at a more rooted-Android standard place: 

cd /system/xbin
busybox mount -o remount,rw /system
cp /data/data/eu.kowalczuk.rsync4android/files/rsync /system/xbin/
chmod 755 /system/xbin/rsync
chown root.shell /system/xbin/rsync
busybox mount -o remount,ro /system

Wi-Fi

Make sure Wi-Fi will stay on!
Menu > Settings > Wireless & networks > Wi-Fi settings > Menu > Advanced > Wi-Fi sleep policy > Never (or never when powered)

BackupPC config

My config: create new host in backuppc web interface with: 

   XferMethod = rsync
   RsyncShareName = /data/, /system/, /mnt/asec/, /mnt/sdcard/
   BackupFilesOnly = not used
   RsyncClientPath = /system/xbin/rsync
   RsyncArgs: add = --one-file-system
   Optional: Any excludes for files you don’t need to back up.

Note that in the mentioned link he's using RsyncShareName = / and playing with BackupFilesOnly but for me it looks like BackupFilesOnly was not respected, so I preferred to have separate RsyncShareName and enforcing the --one-file-system option
Mmm obviously it won't backup the various /mnt/asec/ mounted containers but the original encrypted container should be in one of the other partitions, no?? I've to understand better that APP2SD stuff...
I had errors "Ping too slow" so I increased 

   PingMaxMsec = 400

as anyway it's on local network

Retrieved from "https://wiki.yobi.be/index.php?title=Android&oldid=7336"