2014-03-14

Great news Mac and iOS developers! You don’t have to upload screenshots manually to iTunes Connect anymore. There’s an Apple-provided tool to help you! It kinda sucks, but that’s OK. At least we can script it.

iTMSTransporter stands for iTunes Music Store Transporter, which means it wasn’t made for you. Actually, I don’t think it was made for anyone; my best guess is that it’s a byproduct of the original iTunes Store development ten years ago, and at some point a manager found out about it and realized it could be used by the Content Providers, where “Content” really means “thousands of hours of HD Video”.

It’s really a weird piece of software: it’s a java command-line tool, which is in fact a WebObjects app running locally. It might be able to run on windows somehow, which partially explains this odd choice of technology.

iTMSTransporter is probably already installed on your mac. It’s embedded inside the Xcode bundle: Xcode App submission and Application Loader both use it under the hood. You can also download it directly from Apple’s servers. At 71MB, it’s a rather large download for what it does, but keep in mind it’s a full-blown WebObjects app. Another weird fact: it comes as a .pkg installer that is not signed. For an App Store management tool, I find this particarly ironic.

Anyway, here it is, right inside Xcode:

/Applications/Xcode.app/Contents/Applications/\
    Application Loader.app/Contents/MacOS/\
    itms/bin/iTMSTransporter

You’ll probably want to alias iTMSTransporter= that thing above.

iTMSTransporter: the CLI

The official documentation for the iTMSTransporter tool can be found in the “Manage Your Apps” section of iTunes Connect, under the “Transporter User Guide” link. It’s just a 9 pages PDF, and it very briefly describes the few options relevant to app developers.

We’ll just run iTMSTransporter with no argument, and after a second or two of WebObjects hard thinking, we’ll get the usage info:

usage: iTMSTransporter [-help <arg> | -info | -m <arg> | -version]   [-o <arg>] [-v
    <arg>]  [-WONoPause <arg>] [-Xmx1024m]
iTMSTransporter : iTunes Store Transporter 1.7.9
 -m <arg>         The -m option specifies the tool's mode.  The valid values are:
                  verify, upload, provider, diagnostic, lookupMetadata, status,
                  statusAll, queryTickets, generateSchema, transferTest,
                  downloadMetadataGuides, iTunesExtraQCDownload, iTunesLPQCDownload,
                  iTunesExtraTemplate, iTunesLPTemplate, listReports, requestReport
 -o <arg>         The -o option specifies the directory and filename you want to use
                  to log output information.  By default, Transporter logs output
                  information to standard out. If you specify a filename,
                  Transporter logs the output to the specified file, as well as to
                  standard out.
 -v <arg>         The -v option specifies the level of logging.  The five values
                  are: off, detailed, informational, critical, eXtreme.
 -version         The -version option should be used by itself and returns the
                  version of the tool.
 -WONoPause <arg> The -WONoPause option is only valid on Windows and its value can
                  be 'true' or 'false'.  If an error occurs during script execution,
                  the process idles because the message 'Press any key...' is
                  displayed on the console and the system awaits a keypress. To
                  avoid this behavior, set this property to true
 -Xmx1024m        Specifies that you want to change the Java Virtual Machine's (JVM)
                  allocated memory by increasing the JVM heap size.  By default,
                  Transporter uses a 512MB heap size. You can use the -Xmx1024m
                  option to specify a 1-gigabyte (GB) heap size. Apple recommends,
                  if needed, increasing the heap size to 1024MB by specifying the
                  -Xmx1024 option and adjusting as needed.

(Just look at the -WONoPause and -Xmx1024m options for a minute. What year is this again?)

The only option we care about is -m for Mode. And the only modes we need are:

  • lookupMetadata: download the .itmsp stub package of an app;
  • verify: check if the contents of the modified package if valid before uploading it;
  • upload: upload it back to the iTunes servers.

That’s how iTMSTransporter works: download the package metadata, tweak it and add our screenshots, then upload it back to iTunes.

The .itmsp package

An .itmsp package is simply a folder containing a metadata.xml configuration file, and a number of associated resources (in our case, App Store screenshots.).

File.itmsp/
     * metadata.xml
     [additional resource files]

Although the doc for iTMSTransporter is very short, the package format and the metadata xml is described at length in the Metadata specification, which is once again available in the “Manage Your Apps” section, under the “App Metadata Specification” link.

Let’s start by downloading the metadata for our app:

$ iTMSTransporter -m lookupMetadata\
    -u [iTunes Connect user name]\
    -p [iTunes Connect password]\
    -vendor_id [App SKU]\
    -destination [local download path]

(The App SKU is whatever we set in iTunes Connect. Also, you don’t really have to pass the password as an argument, iTMSTransporter will prompt you if it is omitted.)

As its names implies, lookupMetadata only fetches the metadata.xml file: the current App Store screenshots are not in the downloaded package.

Let’s take a look inside this metadata.xml file.

<package xmlns="http://apple.com/itunes/importer" version="software5.1">

 [snip] Header info

 <software>
  <software_metadata>
   <versions>
    <version string="8">
     <locales>
      <locale name="fr-FR">
       <title>Very Nice App</title>

       [snip] French description and release notes

       <software_screenshots>
        <software_screenshot display_target="iOS-3.5-in" position="1">
         <file_name>fr-01_SignupScreen-2x.png</file_name>
         <size>60846</size>
         <checksum type="md5">f48951ba3076af6022c9e7dba70a20d9</checksum>
        </software_screenshot>

        [snip] More screenshots

       </software_screenshots>
      </locale>
      <locale name="de-DE">

      [snip] German texts and screenshots 

      </locale>
      <locale name="en-US">
      
      [snip] English texts and screenshots 

      </locale>
     </locales>
    </version>

    [snip] Next "Prepare for upload" version, if any.

   </versions>
   <products>
    <product>

    [snip] Product pricing and availability

    </product>
   </products>
  </software_metadata>
 </software>
</package>

This is essentially the same thing that can be set via the iTunes Connect web UI:

  • Information for all the “current” versions is available. That means the live version on the appstore, as well as the next version, being prepared on iTunes Connect.

  • Inside both versions, we find another level of hierarchy, for each locale that’s enabled on iTunes Connect. As you can see, this app is localized in French, German and English.

  • For each locale, the various texts (title, keywords, description, and release notes) are available, as well as the full list of screenshots.

After the versions, the metadata also contains the iTunes Connect products, beginning with the app itself and including any in-app products. Here you can add products, setup pricing and availability. This would deserve another tutorial; in a free app, there’s not much to see here. Just be aware that you can configure pretty much everything related to IAP right from this xml file.

It’s important to note that unmodified fields can be omitted when uploading an .itmsp package. This means, in our case, that we’ll remove the “Ready for Sale” version info, as we don’t want to mistakenly change its description or release notes. We’ll also get rid of the keywords, description, and releasenotes xml nodes, as we just want to deal with the screenshots.

Here’s the data for a single screenshot again:

<software_screenshot display_target="iOS-3.5-in" position="1">
 <file_name>fr-01_SignupScreen-2x.png</file_name>
 <size>60846</size>
 <checksum type="md5">f48951ba3076af6022c9e7dba70a20d9</checksum>
</software_screenshot>
  • display_target, obviously, refers to iPhone4/iPhone5/iPad screen size. If you mess up with the image size, iTMSTransporter will stop you.
  • position lets you control the ordering of the various screenshots. Again, you can’t include more than five images, and you have to be coherent, otherwise iTMSTransporter will complain.
  • file_name, obviously, refers to the name of the screenshot file. This is really a file name, not a path: the files must actally be located at the root of the itmsp package, not in a subfolder, and certainly not outside the package.
  • size and checksum. iTMSTransporter tries to be nice with you: if the size and checksum are identical to the previous version of the file, it won’t be transferred.
    However, you have to compute the size and the checksum yourself, and you can’t put phony values. iTMSTransporter actually checks the values in the xml against the actual files sizes and hash, and will throw an error if they don’t match. I suppose that it makes sense for Video and Audio Content Providers, who want to make sure they upload the correct files, but for screenshots, it’s more of a nuisance, and we’ll have to automate it.

Scripting it

Now that we know how to fetch the .itmsp package and how to set it up, let’s make it all work!

First, we’ll cleanup the metadata.xml file as much as possible, and remove all the entries we don’t want to change. As I mentioned earlier, that means everything but the screenshots.

Next, we have to match the filenames in the xml with the actual screenshot files. Whether the screenshots were automatically generated by test scenarios, or if they were hand-crafted, remember the files must be placed right inside the package:

VeryNiceApp.itmsp/
    * metadata.xml
    * fr-01_SignupScreen-2x.png
    * fr-01_SignupScreen-568h-2x.png
    * fr-02_SearchForm-2x.png
    * fr-02_SearchForm-568h-2x.png
    [...] all FR, EN and DE screenshot files...

And in metadata.xml:

<locale name="fr-FR">
 <software_screenshots>
  <software_screenshot display_target="iOS-3.5-in" position="1">
   <file_name>fr-01_SignupScreen-2x.png</file_name>
    [...]
  </software_screenshot>
  <software_screenshot display_target="iOS-4-in" position="1">
   <file_name>fr-01_SignupScreen-568h-2x.png</file_name>
    [...]
   </software_screenshot>
   [...]
 </software_screenshots>
</locale>
[...other locales...]

We have to update the size and checksum values to match the values of the files. Here comes the script, which probably can be improved: 1

ITCPACKAGE=[path to your .itmsp package]
ITCMETADATA=$ITCPACKAGE/metadata.xml

for FILENAME in `ls $ITCPACKAGE/*.png | xargs basename`
do
    FILEPATH=$ITCPACKAGE/$FILENAME
    
    # Remove all non-essential data from the PNGs
    pngcrush -rem alla -rem text $FILEPATH $FILEPATH-crushed &> /dev/null
    mv $FILEPATH-crushed $FILEPATH

    # Compute the md5 of the file and update its value in the xml file
    xml ed -L -u //_:version//_:software_screenshot[_:file_name=\"$FILENAME\"]/_:checksum -v `md5 -q $FILEPATH` $ITCMETADATA
    # Do the same for the file size
    xml ed -L -u //_:version//_:software_screenshot[_:file_name=\"$FILENAME\"]/_:size -v `stat -f%z $FILEPATH` $ITCMETADATA
done

The pngcrush phase makes sure that pixel-identical files are actuall the same on disk. This will be helpful if you add the itmsp packages to your source repository. Additionally, if you need to overlay some text over the screenshots, or embed the screenshot in an iPhone shape, you’ll probably need to add some imagemagick compose steps.

All that’s left is uploading it back to Apple:

$ iTMSTransporter -m verify\
    -f [path to App Store Package]\
    -u [iTunes Connect user name]\
    -p [iTunes Connect password]

$ iTMSTransporter -m upload\
    -f [path to App Store Package]\
    -u [iTunes Connect user name]\
    -p [iTunes Connect password]

Voilà! Look Ma, no iTunes Connect!

Actually, you’ll want to check the screenshots were successfully uploaded – don’t panic if they don’t show up immediately, there’s a small delay before the new screenshots appear on the Web UI.

This proved very convenient for us at Capitaine Train: we do this at every app update, just before uploading the binary. That way, we’re always sure that the App Store screenshots are always up-to-date with the UI tweaks we’ve added in the latest update. In fact, it has turned the very tedious task of uploading screenshots into a minor step of the Release Checklist.

Going Further

I haven’t (yet) tried to analyze the actual HTTP traffic. I have a feeling it would be easier to reverse engineer the webservices rather than interfacing and parsing the command line. Let me know if you ever look into that :)

  1. You’ll need xmlstarlet: brew install xmlstarlet ↩