On “upgrading” firmware when tricked out

Important note: I wrote this back in September 2008 when upgrading to FW 2.2 but never published it to the blog.  Much of it is outdated, but many of the same principles apply.  Since a lot of people will be upgrading to FW 3.0 if they haven’t already, and I don’t have time to completely rewrite this at the moment, I thought it would still be helpful.

Please see several of the most recent blog posts for information relating to FW 3.0, as well as the rewrites of the main pages.

Situation normal

So you have tricked out your jailbroken iPhone and want to upgrade its firmware as painlessly as possible without losing anything.  If you’re like me, you have both free or purchased App Store apps, cracked IPAs, and a bunch of stuff installed with Cydia and/or Installer.  You also probably have hidden apps and/or are using Categories.

What to do?  There are a lot of guides out there for this process.  In particular, see:

(#1). BigBoss’ “How To Upgrade In iTunes”

(#2). BigBoss’ “How To Restore Your 2.1 Backup”

(#3). modmyi’s “Upgrade from Jailbroken 2.0.2 to Jailbroken 2.1 without losing ANYTHING!!”

But these guides are contradictory or incomplete.  I will not try to claim that what follows is in any way definitive. Instead I will point out some considerations I had when reading over the ones above.  Then I will sketch out a re-do of the steps in BigBoss’ guide #1.

The Unlocking Chimera (or, keeping options open)

We know that as of October 2008, the iphone-dev team was still making “slow progress” on an unlocking solution for the iPhone 3G.  You will recall that unlocking means being able to use your iPhone with any SIM card in the world.  It is not the same thing as jailbreaking.  This is important to me.  In order to maximize the chances of eventually being able to unlock my phone, I did not want to use QuickPwn, which, since you must upgrade your phone in iTunes first, means that the baseband firmware will also be upgraded to 2.1.  We wanted to keep the original baseband firmware while upgrading to 2.1.  This required using Pwnage Tool 2.1 and not QuickPwn.

Most easily retaining everything

Several of the comments about (#1) talks about using the Mac OS X application iBaReS (formally iBackup) to backup your iPhone and restore after an upgrade.  This is a more complete way than using iTunes.  Unfortunately it has some quirks and limitations.  These are addressed below.

Dealing with cracked IPAs

(#1) says nothing about the whole issue of Mobile Installation framework patching to allow cracked IPAs to work. Note that Firmware 2.1 needs an upgraded Mobile Installation framework if you were previously using 2.0.x.

Comments and revisions to BigBoss’ steps in (#1):

First, these are amendments and comments on BigBoss’s steps given my considerations above.  Refer to Guide (#1) for complete details.   I will also underscore this:  “never use ‘upgrade’ on a jailbroken device…Restore, do not upgrade.”

Step 0) Same. Upgrade to iTunes 8.

Step 1) Same. Install & run apt backup.

Step 2) Instead of backing up with iTunes, use iBaReS in individual backup mode with all options checked.  First, review the iBaReS readme and usage scenarios file, and check out this thread.  I found as some others did that running an individual backup first before ever using the standard backup put iBaReS into an infinite loop while checking RSA keys and had to be force quit.  The solution, though sucky, is to let it do a standard backup first.  This of course doubles the backup time, so be prepared for this.  Hopefully this problem will be fixed sooner rather than later.

Step 2B) iBaReS even in full mode does not backup anything in your /User/Media/ folder, which means you need a way to back up anything important to you there. In my case this included /User/Media/Comics/ and /User/Media/ROMs/.  You could just copy files with CyberDuck via SFTP or you could try what I did, the terminal command rsync.  Of course you need to adjust the IPs and the paths:

rsync --exclude iTunes_Control/ -av root@192.168.0.203:/private/var/mobile/Media/
/Volumes/Disk/Archives/iPhone-Media/ --delete

The above command is one line separated by a space, but has been broken into two here for readability.

Steps 3 – 5) Instead of restoring the firmware through iTunes and then jailbreaking with Quickpwn in step 5, we used Pwnage Tool 2.1 in Simple Mode.  I made the mistake of answering “Yes” to the question about whether this iPhone had been pwned before.  Apparently, I should have said “No” even if it had, and let Pwnage Tool guide me in putting the phone into DFU mode.  As a result, I got the dreaded error 1600. Apparently I was in recovery mode and not DFU mode.  Heelfan71’s answer in this thread helped us recover from this, and achieve a successful restore of the custom firmware.

Step 4) Same.  Activate service in iTunes but do nothing else EXCEPT despite what BigBoss says, when you are given the chance to name your device, give it the old name.  I made the mistake of following BigBoss on this one and did not do that.  But after I restored from the iBaReS backup in step 7, iTunes would no longer recognize the iPhone.  No amount of re-installing iTunes 8 or restarting phone or computer solved that, so I was forced to start again with step 3 after making it through step 7.  This time naming it allowed iTunes to recognize the iPhone after step 7.

Step 5) Not applicable because we have already jailbroken in step 3.

Step 6) Same. Using Cydia, install aptbackup… Etc.  Unfortunately, whatever aptbackup did was not restored in step 7 either, so I chose to manually reinstall a bunch of stuff from Cydia.  For me this included: OpenSSH, BossPrefs, Categories, and Cycorder.

Step 6B)  Since you will be using your Wi-Fi connection, make sure you have entered any Wi-Fi password in general settings.  Set auto-lock to never, since we’ll need to restore things in step 7.

Step 6C) One of the things you need to make sure to do at this point is to install the new Mobile Installation Framework IPA crack.  One way to do this easily is to join xsellize and then add to your sources the xsellize repo. (http://xsellize.com/cydia/user-pass/) Then you can install IPA Prep 2, found in the “X Apps” section of Cydia.

Step 7) Instead of restoring from the iTunes backup, we’re going to restore from the iBaReS backup. Again, see the iBaReS usage scenarios document and read this thread.  In summary: 1) restore Applications ONLY with “update mode” checked, then 2) restore user and mobile data WITHOUT update mode.

Step iTunes will restore your settings, contacts, notes, pictures, and all your apps. Now to restore the rest of your Cydia installs. Open aptbackup and hit restore. It should reinstall all your packages from Cydia. If aptbackup is now hidden (missing icon) after your iTunes restore, load BossPrefs, go to hidden icons, and unhide it. You can also unhide any other hidden icons here.

Step 9) If you had a bunch of category folders and moved most things into it, the iTunes backup will have made these icons disappear. You  can unhide them via BossPrefs.

3GS jailbreakers need this

Those lucky enough to have a 3GS who intend to jailbreak it must find their iBSS certificate before jailbreaking it for the first time.  Apparently they must keep this certificate around for future upgrades & jailbreaks.

The process is explained by iClarified.

The more things change…

…the more they remain the same.

Much has changed since I first put together this blog for firmware 2.0, and it became apparent that a lot needed to be updated for firmware 3.0.  In most cases, only the names have changed.  In some cases, like tethering and installing WeDict dictionaries, things have changed more fundamentally.

I’ve now gone back through old blog postings and updated the information accordingly.  Please be aware that everything may change again with any new iPhone OS release, but hopefully this blog should provide a clear enough general road map to remain relevant for some time to come.

UltraSn0w 3.0 unlock completes the cycle

The Dev Team has released UltraSnow, the software unlock for iPhone OS 3.0 that replaces yellowsn0w.  Yellowsn0w is the originary sn0w that started the pwnage-oriented joke of color-coded territorial piss marking.  Cute, and funny, but rather confusing at times since each snow color release actually accomplished different things.  (See key terms here.)

This release completes the hat trick of releases over the past week that update the jailbreak tools for OS 3.0.  First, Pwnage tool was released for Mac OS X, then RedSn0w was released as a cross-platform successor to QuickPwn, and now UltraSnow makes carrier unlocking possible on OS 3.0.

The Dev Team is owed great gratitude by all the jailbreakers out there for their quality work.  So thanks!  Although days behind their own announced release ETAs, the cycle is now complete.  Watching the tens of thousands of comments on the Dev Team blog, it became abundantly clear that people were waiting for each release with a lot of impatience and frustration, many having upgraded to OS 3 before a jailbreak was available, or while needing the carrier unlock to use their phones, despite warnings to wait.  It also became clear that many people never understood the distinction between jailbreaking and unlocking.

UltraSn0w is available via Cydia after adding repo666.ultrasn0w.com to your sources.

Restoring application settings from rsync backup

We all know that iTunes will generally restore your application settings and documents after an upgrade. But if you are jailbreaking and restoring fresh, this rarely goes very smoothly. I prefer to have iTunes sync all my apps from scratch and not touch the backup, then restore the documents and preferences for each app.

For Mac OS X, I put together and AppleScript to do this. It is an advanced script that requires some knowledge of shell scripting, SSH, and applescript.

After reading the initial comments, if you want to go ahead, download the compiled application script from rapidshare here. (Because copy & paste wouldn’t compile, I’m just including the comments here. Must download actual script.)

This download is the version for 6/20/2009. (tested successfully with a couple bug fixes.)


(*
RESTORE APP SETTINGS AND DOCS
by metaclam
http://metaclam.wordpress.com
donations accepted via PayPal at web site above
06/19/2009 (see version history at end of beginning comments)

This script application will restore your documents and preferences folders of all iTunes synched "mobile applications" from an rsync backup of your iPhone to a new iPhone restore.

The assumption is that if iTunes re-installs your applications, they will all have new app ids (the hex strings).
So this script finds the new id and restores the documents folder and the library/preferences folder for each application from the old app id location to the new location.

This script is an ADVANCED script and requires some knowledge of shell scripting and applescript, but not TOO much.
Be warned, it does not do extensive error checking, so watch the log, which by default is set to go to ~/Library/Logs/iPhone_App_Settings_Restore.log

DISCLAIMER:
YOU USE THIS AT YOUR OWN RISK.

To set this up:

#1. You will have first had to run a BACKUP using rsync of your iPhone (before upgrading firmware or otherwise restoring) to a set local directory.
#2. You will need to jailbreak your phone, install OpenSSH from Cydia, etc, and then re-sync all your apps with iTunes.
#3. You will also need to make minor edits to these script properties.
#4. You will also need set up remote SSH login RSA keys on your restored phone.

NOTE: in the following examples I use MY iPhone IP (192.168.0.203) and my local backup directory path (/Volumes/Naga/Archives/iPhone-Backup/).

#1. the command to make a backup using rsync:
rsync --exclude private/var/mobile/Media/iTunes_Control/ -av root@192.168.0.203:/ /Volumes/Naga/Archives/iPhone-Backup/ --delete

This will of course take a long time -- maybe hours depending on how much stuff you have on your phone.
This excludes all your iTunes files which will of course be synched by iTunes itself.

#2. Can't help you much with step #2; if you don't know what all that is, this will do you no good anyway.
Remember you must install a patched mobile installation file if you plan to sync cracked apps etc, and follow relavent directions.

#3. You must edit the first 4 properties in this script as described.

#4. SETUP SSH AUTHENTICATION for ROOT on your iPhone

This is the most tricky part. See this page for details:

http://www.webmonkey.com/tutorial/Automate_a_Remote_Login_Using_SSH-Agent

You will do this after you have set up SSH on your new iPhone and adjusted the root password as necessary (ie, with the "passwd" command via ssh)

In summary, in Terminal you must:

a. create your local user SSH key pair with this command:

-- $ ssh-keygen -t rsa

b. when prompted, save the key in your local user directory as such:

Enter file in which to save the key (/Users/USERNAME/.ssh/id_rsa):

c. then copy the file from your local user directory to your iPhone root directory with this command:

-- $ cat ~/.ssh/id_rsa.pub | ssh root@192.168.0.203 'cat >> .ssh/authorized_keys'

(note, you may need to create the .ssh directory in your root home with 'mkdir .ssh' first)

d. enter your root login password (which is alpine unless you changed it with the 'passwd' command).

VERSION HISTORY
=====
06/19/2009: initial release
06/20/2009:
- fixed searching for apps with spaces
- other bug fixes and tweaks
- tested successfully after restore from FW 2.2 to 3.0!
*)