A celebrated feature of Emacs is its rich ecosystem of packages to extend its behavior. A curated versioned set of these packages are included in the distribution of each Emacs release. These packages are referred to as “built-in” and significant consideration is made by the core Emacs development team to ensure their stability and interoperability for that release. But that consideration does not stop there. Although a built-in package can have updates after an Emacs release, the default policy of the Emacs package manager (package.el) is to avoid upgrading built-in packages. However, this policy can be overridden as of Emacs 29.1 with the customizable variable package-install-upgrade-built-in. If this variable is set to t, then built-in packages can be updated. As this change is relatively recent in the overall history of Emacs, this might be news to a number of readers. For such readers, consider yourselves informed. That said, I was definitely surprised by this policy. The rest of this post is a reflection on why I thought that way.

One more bit before reflection: Users who are uncomfortable with allowing the package manager to aggressively upgrade built-in packages can avoid configuration of package-install-upgrade-built-in and instead use the prefix C-u before invoking package-install. The prefix will “special case” behavior as if package-install-upgrade-built-in was set to t for a specific package update.

What led to surprise

My recent work on a Casual feature required a more recent version of Transient ( > 0.6.0) than what was built-in (0.4.3). I had coded the package dependency specification correctly, but there was confusion when a different user tried to actually install and run it, as package.el avoided updating the user’s local install of Transient by design.

Reflecting on User Expectations

Reflecting on the above, I’ve come to some observations about why I felt such surprise on package.el’s default behavior. It comes down to different expectations of trust:

• Lay users (such as yours truly) in 2024 have been conditioned to extend a lot of trust to software updates.
  • Presumption is that diligent software providers use practices to uphold that trust.
• The Emacs core team in designing package.el does not trust the third party package ecosystem.
  • Presumption is that software providers are not diligent in using best practices.

The divide in the expectations of trust by the provider (in this case the Emacs core team) and the consumers (Emacs users) only gets exacerbated as more built-in packages such as Org and Transient have a higher frequency update schedule than that of Emacs releases. Developers such as myself building packages that require updated versions of built-in packages will only become more pronounced over time.

I claim no monopoly on having an answer to resolving these different expectations. But I do think the Emacs community as a whole needs to advance their conversation on what its package ecosystem should look like in the 21st century. Should it evolve to be more like OS and programming language library package managers? Or should it stay ad-hoc and adversarial in trust? (Are those even the right questions?)

Closing Thoughts

Meditating on software ecosystems, the one that I’m most familiar with is developing for mobile app distribution, particularly Apple’s flavor of it. If we think of Emacs core as an OS and the built-in packages as pre-installed software (apps), we see a striking similarity to the update schedule as illustrated in the table below.

	GNU	Apple	Update Schedule
OS	Emacs core	iOS, iPadOS	OS Release
Pre-installed Software	Built-in Packages	Mail, Weather, Calendar	OS Release
Third Party Software Distribution	ELPA (GNU/nonGNU), MELPA	App Store, Alternate	Publisher Release

All the concerns Apple has faced in building its ecosystem I see are the same ones facing Emacs with its package ecosystem. This is not to say Emacs must mimic Apple here, but rather the questions of trust are the same. It will be interesting to see how the Emacs community answers them.

emacs