Application Extensions

Ben Goodger <ben@mozilla.org>

Abstract

Introduction
A New Take On Extensions
Conclusion

Introduction

Mozilla's two new applications, Firebird and Thunderbird, support user- installable optional components called extensions that adapt or enhance application functionality in various ways. Extensions are easy to create, usually tailored to a specific task, usually written in JavaScript and XUL, and as a result are mostly small, managable compnents.

Nonetheless, extension authors and users have uncovered some problems with the current setup. These include:

Lack of an uninstall capability - Extensions once installed cannot be automatically uninstalled by the system, only disabled. This results in cruft in the Extensions list, and in the install folder.
Extensions overwrite critical files - some Extensions overwrite critical application files with their own versions, causing problems and incompatibilities when users update other parts of the application. The classic example of this is Tabbrowser Extensions, which constantly breaks in nightly builds (basically any time the file that it overwrites changes from under it).
Extensions incompatible between application versions break the application - when you upgrade from Application vN to Application vN+1, code changes in the application can cause extensions to stop working, however the extensions are still loaded despite unknown compatibility.
Difficulty installing extensions into non-browser applications - Users of applications that aren't a browser, like Thunderbird, have a difficult time installing extensions since there is no easy way to install from the browser. Most often the user is forced to download the extension through a browser as a XPI file and then use cumbersome UI in the target application to actually install it.
Extension authors must place files in system locations - Certain file types, such as XPCOM components and preferences defaults files must be placed in specific locations in order to be recognized by the application framework. This makes identification and removal of extension components more difficult.
It is difficult to manage and maintain installed extensions - Given the nature of extensions, new versions are constantly available. It is often difficult or cumbersome to keep track and update installed extensions. Users must manually identify and install newer versions of extensions, which may or may not be compatible with the current application version.

These changes call for a series of changes to the way extensions are handled.

A New Take on Extensions

Goals

To make it easy to install compatible extensions in any new Mozilla app from any web browser.
To make it easy to uninstall extensions completely.
To maximize flexibility for extension authors in the types of files that they can provide.
To prevent outdated extensions from breaking the application.
To provide a scheme to keep extensions and other application components up-to-date.
To design a system to facilitate the above that is forward compatible.

Installing Extensions From A Browser

Currently extensions for all browsers bear the xpi file extension and are identified as application/x-xpinstall. There is no discrimination between intended target application.

The proposed solution is to create a variant of the xpinstall content type for each extension consumer, like so:

application/x-xpinstall;type=extension&app=firebird application/x-xpinstall;type=extension&app=thunderbird application/x-javaarchive;type=skin&app=firebird application/x-javaarchive;type=skin&app=firebird

When an application is installed, it creates a listing in the system MIME registry associating itself as a handler for that type.

For example, when Firebird is installed on Windows, it creates several keys in the Windows Registry - one associating application/x-xpinstall;type=extension&app=firebird with the Firebird executable and another associating an extension (e.g. .fbext) with that content type. This means that the file will be appropriately handled not only when invoked via a link click from any browser (Firebird, IE, Opera etc), but also when invoked from the system shell (e.g. a double click on a .fbext file in Explorer).

The effect on extension authors is that they would need to give distinct file extensions to their extension installers, as defined by each application. Distribution sites would need to send the correct content-type for each extension target application.

Supporting Uninstall

There have been proposals restricting extensions to one file or one specific set of files in order to make extension uninstallation easier. The proposal here is that any xpinstall install operation should be reversable via a log that is created when the xpi file is installed.

Every operation performed in a xpi's install.js file should append an entry to the log, which creates a manifest that can be used to uninstall the xpi later on. Files can be removed, backups restored (when files are replaced), registry keys unset and such like.

This proposal does not go as far as dictating where this log would be stored or what format it would assume, although it is intended that the log would be atomically written after the successful completion of each xpinstall command, and that this would be built into the xpinstall implementation of the various install commands.

Little or no change would be required by the extension author. If the install log were kept with the installed files (which may be necessary in multi-user systems) then an additional function call at the commencement of the installation may be necessary to identify the log file.

Keeping Extension Files Together

To rectify the problem of extensions overwriting or tampering with core system files, extension files should be stored in a location separate from other application files. The proposed structure is:

For Application Directory Extensions:

bin/extensions/<extension_name>/chrome/ bin/extensions/<extension_name>/components/ bin/extensions/<extension_name>/preferences/

For Profile Directory Extensions:

<profile_dir>/extensions/<extension_name>/chrome/ <profile_dir>/extensions/<extension_name>/components/ <profile_dir>/extensions/<extension_name>/preferences/

The component registry and preferences systems are updated to scan for components and defaults files in these locations for installed and enabled extensions.

This mechanism gives extensions full flexibility into the kind of functionality they can provide.

Directory Scan Path Precedence - One question raised by the above proposal is the precedence in which files are loaded. In order to prevent extension dlls from clobbering core application files, the precedence would seem to be: System Components, Profile Extensions Components, System Extensions Components.

Extension Versioning

To prevent extensions designed for older versions of applications from causing problems, they should only be loaded if the application version falls within the range supported by the extension. An extension can identify itself as supporting a range of versions by using the minAppVersion and maxAppVersion properties on its package contents.rdf chrome registry entry. The version scheme works like this:

MAJOR_VERSION.MINOR_VERSION.MAINTENANCE_VERSION

For example, 0.6.1 (maintenance update to 0.6) is less than 0.8 (minor version 0.8) is less than 1.3.1 (maintenance update to 1.3). A special case is for extensions targeting trunk development builds between releases, in which case the appropriate version string assumes the form:

MAJOR_VERSION.MINOR_VERSION+

+ indicates trunk development following MINOR_VERSION, e.g. 0.7+ indicates trunk development after 0.7

When the application is started, the extensions list is enumerated and if the app version falls outside the supported range of any extension that extension is disabled (and the update mechanism is invoked). If the extension is disabled for this reason, an annotation is placed in the extension's entry in the chrome registry preventing it from being enabled in this version, and identifying the extension as being disabled because of a compatibility problem.

Extension Updating

Extension Updating is a mechanism for determining if there are newer compatible extension versions available from the web. This is useful if either the user wants to stay on the cutting edge of their favorite set of extensions, or the user updates to a newer version of the application and the old set of extensions needs to be updated. These two use cases are outlined below.

Manual Invokation

The user will be able to periodically check for updates to the application and installed extensions, or check on demand. When one of these events occurs, a list of installed extensions and their versions is sent to the extension server and the server matches those with the versions it has available. If newer versions are available it responds with a list of available updates. The current application version is sent with the list to ensure that the retrieved list is compatible.

Compatibility Invokation

When a new version of the application is installed, all incompatible extensions are annotated as described above. This list is sent to the server as in the Manual Invokation case, the user is notified and given the option to visit the extension server where he or she will be presented with a list of available upates to install.

As long as there are incompatible versions in the chrome registry, the system will periodically check the extension server for new, compatible versions.

Note - if the incompatible version installed is of the same extension version number as the compatible version on the server, no download is performed, the local incompatible version's maximum application version value is incremented to that of the compatible version from the server's. This is for the case where an old extension continues to work with a newer version of the application, and the author simply updates its compatibility information on the server.

Web Front End - User Level

The Web Front End is designed to show extensions for Mozilla applications that are compatible. The Web FE will be part of a larger software update mechanism that will be hosted on update.mozilla.org. The Web FE will at minimum have the following elements:

A menu listing applications that extensions are available for, defaulting to Firebird.
In the Firebird section, if the user is using Firebird, a list of extensions compatible with that installed version, alternately a selector is presented defaulting to the most recent release that filters the list. This setting is remembered with a cookie.
Extension entries in this list, composed of Extension Name, Extension Author, Extension Homepage, Extension Version, Extension Description, Extension File Size, Extension Last Update Date, Extension Install Icon, Compatibility range (see above), and an install link. If the user is using Firebird to install an extension into Firebird, the XPInstall link will be a XPInstall js api link featuring the Extension icon as metadata, otherwise it will be a link to the extension file, which will be handled by the system's native content type dispatch. (e.g. .tbext invokes Thunderbird. See above.)
The Web FE must be capable of showing a list that only includes a named set of extensions, as required by the Update section above. For example, if the user has extensions A, B and C installed, and chooses to check for updates, if B and C have newer versions available, a query to the update server should result in a list containing the newer versions of B and C being shown. This will also be used in the compatibility check when a new version of an application is installed, see above also.

The Web FE will optionally (hopefully) have these elements:

Categories - categories of extension for each application. An extension might fit into more than one category. Categories can be used as filters on the extension list.
Ratings - a rating mechanism that lets users rate an extension using a graphical meter out of 5 stars.
Forums - each extension should have its own mini forum for user comments.
Search - the text of the extension name and description text at least should be searchable.
Best Rated - Sort by best rated, most popular, etc.
Essentials - Editorial team features best extension.

Important - the important areas of difference between this system and the current implementation on extensionroom are that the list shown is for one app only at any given time and that the list shown is only of extensions compatible with a specific version.

Web Front End - Author Level

Each extension author will have his or her own account with the extension service. Rather than being maintained by a central authority the extension list will be updated by authors. Approval by a moderator may be required for new account requests.

The system should allow authors to create an account with an email address and a password, and their name. They will then be able to add as many extensions as they want to this account, each extension will appear in the list for the applicable application.

When adding an extension, the author will be asked for the following information:

Extension Name (string)
Extension Version (version_string)
Target Application (enumeration value, pick from list)
Target Application Version Range (from: version_string to: version_string)
Extension Homepage URL (string)
Extension Install Icon URL (string)
Extension File (string via File Upload Control)
Categories (check list)

The user will be able to see a list of his or her extensions when they log in, and will be able to edit details. For instance, if a new version of the application is released and extension X is still compatible with the new version, the author will be able to log in and increase the upper bound on the Target Application Version Range field so that users of the newer application version will be able to continue using compatible older versions of extensions.

The advantages of this author-based system is that the author is empowered to make changes and update extension files when they become available. Also, the author can maintain several versions of the same extension for different versions of the application (e.g. 1.0 vs. 1.3+)