It's kind of a pain to have different behavior when you're debugging.
Some errors may not present without them, so best to remain predictable
and permit these optimizations even when debug mode is on.
defcustom does some initialization work to accommodate the possibility
that the user has set the variable before it was defined. This work is
unneeded so early at startup, so I disable it (temporarily).
In the future, Doom will use defcustom more, as it's a helpful
indication to readers what variables I intended for configuration (and
helps with discovery of options through `M-x doom/help-custom-variable`
or `M-x customize`). As that transition occurs, the benefit of this
optimization will begin to show, but for now its effect on startup time
is negligible.
* lisp/doom.el (warning-suppress-types): set this immediately. Since its
default value is nil and this happens so early at startup, we don't
have to be considerate of defaults. Plus, this custom-dont-initialize
optimization can cause breakage if a warning is thrown *before* before
this setting is changed.
Where f9201eb introduced a general context system, this one introduces
one for modules, to simplify our let-bind game when interacting with
modules, and to more efficiently expose module state to modulep! (which
gets called at runtime a great deal, so its performance is important).
* lisp/doom-lib.el (doom-log): simplify macro and introduce
doom-inhibit-log variable.
* lisp/doom-modules.el (modulep!): fix reported file path if modulep!
fails to find the local module.
* lisp/lib/debug.el (doom-debug-variables): disable doom-inhibit-log
when debug mode is on.
Ref: f9201eb218
load! effectively loads (file-name-concat (dir!) PATH) which, in this
case, is concatenating two absolute file paths. Emacs does the right
thing and loads PATH, but I don't want to rely on this good fortune as
it could be broken in a future update.
Introduces a system to announce what execution contexts are active, so I
can react appropriately, emit more helpful logs/warnings in the case of
issues, and throw more meaningful errors.
* bin/doom: load module CLIs in the 'modules' context.
* lisp/cli/doctor.el: load package files in 'packages' context.
* lisp/doom-cli.el:
- (doom-before-init-hook, doom-after-init-hook): trigger hooks at the
correct time. This may increase startup load time, as the benchmark
now times more of the startup process.
- (doom-cli-execute, doom-cli-context-execute,
doom-cli-context-restore, doom-cli-context-parse,
doom-cli--output-benchmark-h, doom-cli-call, doom-cli--restart,
doom-cli-load, run!): remove redundant context prefix in debug logs,
it's now redundant with doom-context, which doom-log now prefixes
them with.
* lisp/doom-lib.el (doom-log): prefix doom-context to doom-log output,
unless it starts with :.
* lisp/doom-packages.el (package!, doom-packages--read): throw error if
not used in a packages.el file or in the context of our package
manager.
* lisp/doom-profiles.el (doom-profile--generate-init-vars,
doom-profile--generate-load-modules): use modules doom-context instead
of doom-init-time to detect startup.
* lisp/doom-start.el (doom-load-packages-incrementally-h): move function
closer to end of doom-after-init-hook.
* lisp/doom.el:
- (doom-before-init-hook, doom--set-initial-values-h,
doom--begin-init-h): rename doom--set-initial-values-h to
doom--begin-init-h and ensure it runs as late in
doom-before-init-hook as possible, as that is the point where Doom's
"initialization" formally begins.
- (doom-after-init-hook): don't trigger at the end of command-line-1
in non-interactive sessions. This will be triggered manually in
doom-cli.el's run!.
* lisp/lib/config.el (doom/reload, doom/reload-autoloads,
doom/reload-env): use 'reload' context for reload commands.
* modules/lang/emacs-lisp/autoload.el (+emacs-lisp-eval): use 'eval'
context.
* modules/lang/org/config.el: remove doom-reloading-p; check for
'reload' doom context instead.
* lisp/doom-cli.el:
- reference backport source commit.
- doom-cli--restart: a type check is all we need here. This is a
programmer error, not a user error.
* lisp/doom-editor.el (recentf): mention recentf-show-abbreviated (added in
emacs-mirror/emacs@32906819ad)
* lisp/doom-keybinds.el (doom-init-leader-keys-h): move to
doom-after-init-hook, in case the user customizes leader variables in
a previous hook (like emacs-startup-hook or after-init-hook).
* lisp/doom-start.el: use eval-when! to compile out the section on
non-macOS systems (when Doom gets around to compiling its core files,
later).
* modules/config/literate/autoload.el (+literate-config-file): use
file-name-concat instead of string concat. This relaxes the
requirement that doom-user-dir end in a /; a requirement I intend to
fully phase out.
* modules/lang/emacs-lisp/autoload.el (+emacs-lisp-non-package): remove
empty map! macro in flycheck-emacs-lisp-check-form. The macro already
no-ops at compile-time/in noninteractive sessions since b480ed51a3.
* modules/ui/hl-todo/config.el (hl-todo-keyword-faces): revise
commentary for default hl-todo keywords.
Ref: emacs-mirror/emacs@32906819ad
Ref: b480ed51a3
* lisp/doom-lib.el:
- (fn!): unroll the loop into a single, fast setplist that
doesn't require a cl-lib macro (autoloading which seems to throw an
error on flatpak/snap builds; still investigating that one).
- (add-transient-hook!): Removes a redundant let-bind. `sym` is
already lexically bound outside the function. This will break
anywhere lexical-binding is nil though. Not sure if I should cater
to that scenario...
- Adds doom-module-packages-file and doom-module-metadata-file.
- Uses them and the other doom-module-*-file variables where they were
previously hardcoded.
- Add .el extension to doom-module-{init,config}-file; it is now the
consumer's responsibility to strip/change/keep the extension as they
see fit.
I intend to phase out the internal usage of use-package in Doom's core
and modules. The macro is too complex and magical for our needs.
That said, until we've fully removed it, this :config use-package is
hardcoded to be enabled-by-default, until use-package has been
refactored out of core and modules. It'd be wise not to add it to your
doom! blocks yet.
This introduces a depth field for modules so that they may dictate their
load order explicitly, it also treats depths <= -100 or >= 100 as
special depths, which will be loaded early, before their respective
doom-{before,after}-module-{init,config}-hook. This permits psuedo
modules like :core and :user modules to be treated as normal modules
without too many special cases.
This also fixes a module load order issue on Emacs 29 (#6813), caused by
emacs-mirror/emacs@4311bd0bd7, which changed the return value order of
hash-table-{keys,values} causing modules to be loaded in reverse order;
resulting in the loss of evil keybinds, among other things.
Other notable changes:
- Changes the data structure for module data caches from a list to a
vector. Uses less memory and permits faster lookups. Also adds two
depth fields to the front of it.
- Changes the signature of doom-module-list and doom-package-list.
- Renames doom--read-packages -> doom-packages--read for consistency
with naming convention.
- Add doom-module-depth function.
- Adds a temporary doom-core-dir/init.el file, which is responsible for
loading doom-*.el.
Fix: #6813
Ref: emacs-mirror/emacs@4311bd0bd7
An unintended change snuck into 2c14eff. The :core and :user virtual
modules are no longer stripped from the module list before iterating
through (and loading) them. This meant that Doom would load these two
like regular modules (and first, since these two are always at the start
of the list).
This is harmless for :core, because it has no init.el or config.el, but
:user does! This means $DOOMDIR/{init,config}.el would be loaded
twice (once before all other modules and again afterwards), causing load
order issues (like #6818).
Fix: #6818
Amend: 2c14eff7f1
- Batch more variables in Doom's autoloads files.
- Remove all the register-definition-prefixes calls generated in
autoloads files (for both modules' and packages' autoloads). These
don't serve much purpose, and only incur added cost growing a large
hash table.
Any buffers created before after-init-hook could trigger these hooks,
which may house expensive functions, but never anything that is
important at startup time.
However, it must not occur later than after-init-hook (which triggers
before file arguments are processed and file buffers are created).
To understand this issue, you have to understand these two things:
1. Doom builds an init file which combines all its autoloads (for
packages and modules), and Doom's bootstrapper (which loads modules,
$DOOMDIR, etc). This init file is byte-compiled.
2. When Emacs byte-compiles elisp, docstrings are lazy-loaded (by
embedding them in the elc as commented text to be retrieved later).
This is generally done to save on memory.
Now the issue: when these lazy-loaded docstrings are retrieved, Emacs
may evaluate the whole file to find it, including Doom's bootstrap
process, reloading all its files, the user's config files, and running
all its startup hooks. Not only is this terribly expensive, reloading
these files may have disastrous effects.
One such effect is compounded by Marginalia, which invokes this
docstring fetch process (by calling the `documentation` function in
`marginalia--function-doc`) for *each* symbol in the `M-x` or `C-h
{v,f}` completion lists, which means Doom re-bootstraps multiple times
and rapidly, causing Emacs to totally lock up.
The solution is to simply gate the expensive part of the initfile so it
doesn't run more than once, at startup, and when `doom/reload` is
called. The rest of the file loads instantly.
Still, this is a bit flimsy. I'll think of a more elegant solution
later.
- Since its arguments aren't used, make the advice n-arity, to future
proof the advice.
- Add commentary on load's side-effect on user-init-file.
- Add NOSUFFIX arg to load call, to spare Emacs the file IO of searching
for init.%d.elc{.{so{,.gz},elc{,.gz},el{,.gz},,gz}}.
Due to $DOOMPROFILE being set to an empty string when persisting Doom
CLI sessions, which would affect any case where a CLI command restarts
the session (e.g. when the :config literate module tangles a config or
'doom --debug ...' restarts to set DEBUG=1).
What used to be a `byte-compiled-config` trait, displayed in your `M-x
doom/info`, is now `compiled-user-config`, `compiled-core`, and
`compiled-modules`, for more helpful granularity for debugging possible
byte-code issues.
Due to a technical limitation of Emacs <=28, launching Emacs out of a
non-standard location is non-trivial, and `doom run` tries to promise
that it can do so on demand. Emacs 29 does introduce a --init-directory
switch that would make this easy, but it'll be some time before we can
rely on it.
So 'doom run' creates a fake $HOME in /tmp/doom.run/ and writes a
bootloader there to load your Doom config remotely. But there's a
problem: in this fake $HOME, none of the user's config, cache, data, or
binscript directories are available, so I symlink them there. This
should at least resolve the most trivial incompatibilities (like the
lack of all-the-icons fonts, which typically get installed to
$HOME/.local/share/fonts/ -- see #6807), but there may be yet more edge
cases. Still, this is a good enough compromise for now.
Fix: #6807
In 4a25375, it seemed that only setting the variables to nil early
enough would be sufficient, but this turned out not to be the case.
There's no avoiding calling the mode to disable it.
Ref: 58c0de6841
Amend: 4a253757cb
- Adds $DOOMPROFILELOADFILE: Controls where to read and write the
profile loader. Changing this may be helpful for users on nix/guix,
who have deployed Doom to a read-only location. This sets
`doom-profile-load-file`.
- Changed profile load file's default location (used to be
$EMACSDIR/profiles/init.el, is now $EMACSDIR/profiles/load.el). The
gitignore was updated to reflect this.
- Adds $DOOMPROFILELOADPATH: A colon-delimited list of profile config
files and directories (semi-colon on Windows) which dictate what Doom
reads in order to discover your profiles. Config files are required to
have an *.el extension. This sets `doom-profile-load-path`.
- Changes the nomenclature around this loader script. I used to refer to
it as the profile bootstrapper. I'll now refer to it as the profile
load file, and I've renamed `doom-profiles-bootstrap-file` to
`doom-profile-load-file` to reflect this.
- The variables `doom-profile-dirs` and `doom-profile-config-files` were
merged into doom-profile-load-path.
- Both envvars have also been documented in `doom help` (and
$DOOMPROFILE's has been updated).
Ref: #6794
Particularly DOOMPROFILE, without which the --profile switch wasn't
actually doing anything, and profile sessions would (silently) use the
default user-emacs-directory and doom-user-dir.
- Swap out the funcall+alist lookup for a pcase (which is expanded to a
cond, which is is faster and easier to read).
- Wrap bootstrap file to $EMACSDIR/profiles/init.el, but byte-compile it
to $EMACSDIR/profiles/init.X.el where X is emacs-major-version.
- Make doom-profiles-save's second argument optional (defaults to
doom-profiles-bootstrap-file).
- Make doom-profiles-save throw a error if byte-compilation fails for
some reason.
- Rename the tempvars to include 'doom' in their name, so debuggers know
where they originate.
In this commit I start using doom-profile-*-dir vars, though it will
make no difference to the default Doom install (only to profiles, and in
the future, in v3, where we'll drop $EMACSDIR/.local entirely).
The profile bootstrap file's first form is the doom-version it was
generated with. If this has changed, it should be considered outdated,
even if the user's profiles haven't changed.
If Doom doesn't live in ~/.emacs.d or ~/.config/emacs, then it cannot
play the role of bootloader, so opt out of generating the profile
bootstrappper in this case.
That said, don't disable the profile system entirely; it can still be
useful for internal, noninteractive, and sandbox use.
So that the resulting file expands to less code and doesn't apply too
magic to file paths (it may be unwanted).
And don't try to unintern a lexical binding.
Fixes two issues with implicit profiles:
1. Where user-emacs-directory (and sometimes doom-user-dir) would be
unexpanded in the profile init file (generated by 'doom profiles
sync'), which would be ineffective at runtime.
2. Where an implicit profile with a .doomprofile that lacks a
user-emacs-directory setting would not have any user-emacs-directory
set for it at all. Instead, it should fall back to that profile's
location.
The sudden spike in CPU and memory utilization alarms people, so I've
reduced how many cores native-comp will use. In non-interactive
sessions, it will use all of them, however (that is, when I later
introduce an AOT switch).
You can still override this by setting native-comp-async-jobs-number or
comp-num-cpus yourself.
I use advice instead of setting comp-num-cpus so that users to avoid
trampling on default behavior, or attempts by the user to change them.
It really isn't important if this function succeeds or not, but it seems
its stability is highly variable, dependent on your specific build and
version of Emacs. Since there are a lot of 29 Doom users, best to be
more permissive and simply fall back to byte-compilation if native-comp
fails.
Causing the envvar file to be generated to wrong place, and thus never
be updated/properly loaded at runtime.
This new setting is for later, where I'll integrate the envvar generate
into the profile generator proper.
Where they will be further generalized, later.
This also prevents an issue where org was loaded while the profile init
files are generated, which caused a warning about org-loaddefs which
introduces a noticable delay.
I forgot to add the definitions for the 'doom profile{s,}' commands in
b914830, which causes "unrecognized command 'profiles sync'" errors on
'doom {sync,upgrade}'.
My unparalleled brilliance is 4 parallel universes ahead of me, clearly.
Amend: b914830403
The doom-modules table should reflect the user's intentions, not the
actual state of its modules (e.g. by omitting modules that couldn't be
found at activation time).
Each of these functions have a native-comp guard, which may be overkill,
but for the time being they should be grouped together, to indicate
their relationship.
Moves this from doom-ui to doom-start, since there is more savings to be
had if this is done early.
Also moves the menu-bar fix for macos out of the :os macos module into
doom-start, because it is a fix (and for a Doom optimization) and not a
feature, so it shouldn't be behind a module.
BREAKING CHANGE: This commit makes three breaking changes:
- Doom now fully and dynamically generates (and byte-compiles) your
profile and its init files, which includes your autoloads, loading
your init files and modules, and then some. This replaces
doom-initialize-modules, doom-initialize-core-modules, and
doom-module-loader, which have been removed. This has also improved
startup time by a bit, but if you use these functions in your CLIs,
for instance, this will be a breaking change.
- `doom sync` is now required for Doom to see your profiles (and must be
run whenever you change them, or when you up/downgrade Emacs across
major versions).
- $DOOMDIR/init.el is now read much earlier than it used to be. Before
any of doom-{ui,keybinds,editor,projects}, before any autoloads are
loaded, and before your load-path has been populated with your
packages. It now runs in the context of early-init.el, giving users
freer range over what they can affect, but a more minimalistic
environment to do it in.
If you must have some logic run when all that is set up, add it to one
of the module hooks added in e08f68b or 283308a.
This also poses a significant change to Doom's load order (see the
commentary change in lib/doom.el), along with the following (non
breaking) changes:
1. Adds a new `doom profiles sync` command. This will forcibly resync
your profiles, while `doom sync` will only do so if your profiles
have changed.
2. Doom now fully and dynamically generates (and byte-compiles) your
user-init-file, which includes loading all your init files, modules,
and custom-file. This replaces the job of doom-initialize-modules,
doom-initialize-core-modules, and doom-module-loader, which have been
removed. This has also improved startup time by a bit.
3. Defines new doom-state-dir variable, though not used yet (saving that
and the other breaking changes for the 3.0 release).
4. Redesigns profile directory variables (doom-profile-*-dir) to prepare
for future XDG-compliance.
5. Removed unused/unimportant profile variables in doom.el.
6. Added lisp/doom-profiles.el. It's hardly feature complete, but it's
enough to power the system as it is now.
7. Updates the "load order" commentary in doom.el to reflect these
changes.
Emacs' version library (e.g. version-to-list) understands a number of
suffixes (see version-regexp-alist), but -dev is not one of them. Rather
than break compatibility (or impose a new, non-portable value onto
version-regexp-alist), I think it's best we adopt -pre instead. I
could've chosen -rc, -alpha, -beta, or -git, but I don't think any of
these accurately represent Doom's current state yet (and I don't want to
lock its versioning to git).
The default-frame-alist properties are only necessary for the scrollbar.
The variables are enough for the rest. Also, no need to set
x-gtk-use-symtem-tooltips if we're turning off tooltips anyway.
Also moves the UI config that snuck its way into doom-start back to
doom-ui.
doom-before-init-hook runs before $DOOMDIR/init.el is loaded.
doom-after-init-hook runs at the *very* end of the Emacs startup
process (after window-setup-hook).
BREAKING CHANGE: For consistency and correctness, I've renamed the
module init/config hooks, and added new ones:
- Adds doom-before-modules-config-hook
- Adds doom-after-modules-config-hook (replaced doom-before-init-modules-hook)
- Adds doom-before-modules-init-hook
- Adds doom-after-modules-init-hook (replaced doom-init-modules-hook)
- Removed doom-after-init-modules-hook (replaced w/ after-init-hook)
The old naming (and timing) was counterintuitive. Now, it's named after
the loaded file group (init.el vs config.el), and I added before/after
variants. Altogether, this should make them less ambiguous.
I've also moved some functions in various modules to more correct hooks.
Load order before this change:
- $EMACSDIR/early-init.el
- $EMACSDIR/lisp/doom.el
- $EMACSDIR/lisp/doom-start.el
- $DOOMDIR/init.el
- {$DOOMDIR,~/.emacs.d}/modules/*/*/init.el
- `doom-before-init-modules-hook'
- {$DOOMDIR,~/.emacs.d}/modules/*/*/config.el
- `doom-init-modules-hook'
- $DOOMDIR/config.el
- `doom-after-init-modules-hook'
- `after-init-hook'
- `emacs-startup-hook'
- `window-setup-hook'
Load order after this change:
- $EMACSDIR/early-init.el
- $EMACSDIR/lisp/doom.el
- $EMACSDIR/lisp/doom-start.el
- $DOOMDIR/init.el
- `doom-before-modules-init-hook'
- {$DOOMDIR,~/.emacs.d}/modules/*/*/init.el
- `doom-after-modules-init-hook'
- `doom-before-modules-config-hook'
- {$DOOMDIR,~/.emacs.d}/modules/*/*/config.el
- `doom-after-modules-config-hook'
- $DOOMDIR/config.el
- `after-init-hook'
- `emacs-startup-hook'
- `window-setup-hook'
I move our hackiest and least offensive startup optimizations to core,
so they're easy for me to keep track of (they'll likely change often,
between major Emacs releases), to keep them from affecting non-Doom
profiles, and make it easy for readers to use as a reference.
...that are always enabled. This way, the module API treats them as any
other module.
This also changes doom-module-load-path. If supplied directories,
doom-user-dir will not be the CAR of its return value. If no dirs are
supplied, then doom-core-dir and doom-user-dir are included (and will
always be the first two items in the returned list).
For small amounts of data, symbol plists are the most efficient (space
and time wise) as data access gets in Emacs. Hash tables, though O(1),
impose a minimum threshold of overhead before it becomes the efficient
option, but this benefit won't be obvious for datasets of at least 60 or
less.
Since modulep! is used *a lot*, and used to determine a module's
state (and state of its flags), there is a benefit to caching it.
Still, this is only a read-only cache, and does not replace the
`doom-modules` hash-table, which will always be the preferred interface
for the rest of the module API.
Used to return the hash-table `doom-modules` (if not all-p), but I've
changed it to return a list of cons cells (:CATEGORY . MODULE),
representing all enabled modules, in the order they were enabled.
The purpose of this change is to prepare for a change in the structure
of doom-modules, and how Doom stores its module metadata.
- Rename doom-module-path -> doom-module-expand-path, to better reflect
its purpose.
- Optimize doom-module-locate-path to try caches and
locate-file-internal, before looping through doom-modules-dirs.
- Rely on file-name-concat to join paths, rather than string
concatenation. file-name-concat is more robust for the purpose and
has lower overhead than expand-file-name.
- Add a 27.x check, recommending that users upgrade to 28.1.
- Removed the check for <27. Loading doom.el with sub-27 will already
fail with a detailed error about supported versions.
- Now detects development (.50) and pre-release (.9x) builds of Emacs,
and warns the user of their dangers.
- Revises the warning for 29+ or .50|.9x users.
It's not useful in those scenarios, and is more likely to throw up
unrecoverably keybind conflict errors, when load order is determined by
an over-eager byte-compiler, rather than Doom's standard startup
process.
The debugger appears to display uninterned symbols properly, and since
these symbols should never be touched/referenced by users, there's no
reason to pollute the obarray with these transient symbols.
Rather than reimplement its face lookup (and have two versions of
doom-print-ansi-alist -- one for 27.x and one for 28+), let's just rely
on ansi-color.
These only benefit interactive sessions, and doom-start's responsibility
is to configure interactive sessions; it doesn't make sense to keep
these in core.
This refactors how Doom captures and redirects its output (to stdout and
stderr) into a more general with-output-to! macro, and:
- Simplifies the "print level" system. The various doom-print-*-level
variables have been removed.
- Adds a new print level: notice, which will be the default level for
all standard output (from print!, doom-print, prin[ct1], etc).
- Adds a with-output-to! macro for capturing and redirecting
output to multiple streams (without suppressing it from stdout). It
can also be nested.
- Changes the following about doom-print:
- Default :format changed to nil (was t)
- Default :level changed to t (was `doom-print-level`)
- No longer no-ops if OUTPUT is only whitespace
This commit reduces the debug log noise, makes it easier to
read/parse/search, and soft-introduces a convention for doom-log
messages, where they are prefixed with a unique identifier loosely named
after it's running context or calling function.
I haven't enforced it everywhere doom-log is used yet, but this is a
start.
BREAKING CHANGE: This removes the doom-incremental-load-immediately
variable. Instead, set doom-incremental-first-idle-timer to 0 to force
all iloaded packages be eagerly loaded at startup. This is already the
default behavior for daemon sessions.
BREAKING CHANGE: If anyone is using Doom's CLI framework and are
defining their own CLIs with any of the following macros, they'll need
to be updated to their new names:
- defautoload! -> defcli-autoload!
- defgroup! -> defcli-group!
- defstub! -> defcli-stub!
- defalias! -> defcli-alias!
- defobsolete! -> defcli-obsolete!
These were renamed to make their relationship with CLIs more obvious;
they were too ambiguous otherwise.
Necessitated by 7e0c2ed, and missed in 45a66cd. This would indirectly
cause "No :repo specified for package 'X'" errors for packages with a
`:local-repo` relative to their packages.el file.
Amend: 45a66cda60
Ref: 7e0c2ed8a3
In v3, doom-module data is stored in symbol plists, but in v2, it's
stored in a hash table. Some v3 code snuck into 45a66cd, which made Doom
try to read module data from plists that hadn't been initialized yet, so
Doom could no longer see your module settings.
Fix: #6769
Amend: 45a66cda60
Otherwise, doom-module-from-path (and modulep!) would fail to detect the
module they're in, or at least, modulep! would incorrectly return nil,
even for enabled modules.
This issue is what would've caused the package list or the doctor to
include/consider packages in disabled modules or behind disabled flags.
Writing a debugger for Elisp is too much hassle. `debug` itself isn't
very customizable without a *lot* of boilerplate, so instead of writing
my own, it's more effective to advise debug instead. Certainly, I don't
do anything with it yet, but I will soon.
Emacs 27.x does not collapse consecutive slashes in a file path when
trying to load them, and instead discards everything before it and
treats the rest as an absolute path, e.g. "~/some//path/foo/" ->
"/path/foo". This is not the case in 28.1, but Doom's backport of
file-name-concat did not take this into account, so it's been modified
to trim trailing slashes.
Fix: #6766
Amend: 433c9e344d
This is caused by a bug in recent builds of Emacs 29, where
`loaddefs-generate` will activate emacs-lisp-mode to read a package's
autoloads, but does so without suppressing its mode hooks. Other
packages may add functions to this hook from their autoloads (like
overseer.el does). Calling these functions will initiate a chain
reaction where other packages will be loaded (plus their dependencies),
but aren't guaranteed to be available so early in the bootstrap process.
The result are file-missing errors about seemingly unrelated packages,
like pkg-info or dash.
Ref: emacs-mirror/emacs@0d383b592c
Fix: https://discourse.doomemacs.org/t/3149
Occurs when a site-file fails to be natively compiled, and Doom attempts
to write an error file in the same directory. On some systems, the
site-lisp directory is in a read-only tree/mount (like nix and guix).
This should suppress those attempts.
Relying on eval-after-load's compiler-macro magic (after 8b4f722) can
cause scope issues with nested macros (like file! and dir! running in
the context of a temp buffer), so best we use with-eval-after-load
directly.
Ref: 8b4f722fa3
To ensure that they're expanded at a file's top-level, while expanded,
where they're used. It also fixes a few inlined uses of the file!
macro (e.g. in `load!`, as reported in #6764), which was prematurely
committed ahead of this change.
Close: #6764
Amend: a179b8d262
b7f84bd introduced a nasty regression that caused an infinite loop and
runaway memory usage on some pgtk+native-comp builds of Emacs when it
attempted to perform deferred native compilation of your packages. This
would make Emacs unusable and, if left alone, could even crash your
system.
The only Emacs builds I'm certain are affected are derived from
flatwhatson/emacs (like emacs-pgtk-native-comp on Guix and Arch Linux in
particular). 28.1 stable and master (on emacs-mirror/emacs@e13509468b)
are unaffected.
It appears that some, earlier pgtk builds stack idle timers differently.
I'm not entirely sure how, because it doesn't manifest in more recent
builds of Emacs, and I'm already burnt out on debugging this, but here's
how Doom encountered it:
Doom has an incremental package loader; it loads packages, piecemeal,
after Emacs has been idle for 2s, then again every 0.75s until it
finishes or the user sends input (then it waits another 2s before
starting again). However, if at any time the iloader detected that
native-compilation is in progress, it waits 2s before trying
again (repeat, until native-comp is done). But here's the catch, given
the following:
(run-with-idle-timer
2 nil (lambda ()
(run-with-idle-timer 1 nil (lambda () (message "hi")))))
I had assumed "hi" would be emitted after 3 seconds (once idle), but
instead it is emitted after 2. Like this, Doom's iloader would elapse
one idle timer directly into another, ad infinitum, until Emacs was
forcibly killed.
By switching to run-at-time and employing my own rudimentary idle timer,
I avoid this issue. Also, the iloader no longer needs to be considerate
of native-comp, because the latter does its own rate-limiting controlled
by native-comp-async-jobs-number.
Amend: b7f84bdd01
This allows us to load them via doom-require. Why not use normal
features? Because Doom's libraries are designed to be loaded as part of
Doom, and will openly rely on Doom state if needed; this is a contract I
want to enforce by ensuring their only entry points are through
`doom-require` or autoloading.
I will add them to the rest of the libraries later.
Site-node: this also adds Commentary+Code to the comment headings, as I
want a space to use that space to describe the library, when I get
around to it.
These functions are light wrappers around require and load, such that
Doom will catch (and decorate) any errors from the target file, and is
also capable of loading Doom's subfeatures.
This is a regression from 948f946, where a bunch of mkdir calls were
removed prematurely. In v3, other processes are responsible for creating
these directories, but those haven't been implemented yet.
Fix: #6756
Amend: 948f9461a7
If doom-emacs-dir contains a "~", attempting to call `git -C` will fail
with an error like:
fatal: cannot change to '~/.config/emacs/': No such file or directory
Fix this by canonicalizing the filename.
I prefer to be more explicit about these variables' defaults, then to
rely on proper load order and unverified global state to ensure they're
properly set.
BREAKING CHANGE: This finally removes 'doom refresh'. It was first
deprecated in 8a77633 and disabled in 8c37928, and has long since been
replaced with 'doom sync'.
Ref: 8c37928de2
Ref: 8a7763337d
And emit more informative errors if they fail.
This eval-when-compile approach is used in preparation for v3, where
Doom's core libraries will be byte-compiled.
This was done to purge superfluous files from Doom's project structure
and simplify its entry points. And with early-init.el now acting as
Doom's universal bootstrapper (see c05e615), we don't have enough
bootstrap logic to warrant being its own file.
Also removes the redundant version check, given doom.el is assured to be
loaded before doom-cli, and performs its own check.
Ref: c05e61536e
A concise alternative to the file IO elisp idioms we're used to,
involving some combination of with-temp-file, with-temp-buffer,
insert-file-contents, coding-system-for-{read,write}, write-region, read
loops, print-to-current-buffer loops, etc.
These were engineered to make reading/writing text and lisp data from/to
files simpler, and will be used extensively in the v3 CLI.
Consecutive expand-file-name and recursive apply's can be expensive, so
the function has been simplified to rely more on file-name-concat. This
does change one trait about it, however: absolute paths in SEGMENTS no
long reroot the whole path, and are concatenated as ordinary file
segments.
The performance benefit is more pronounced on Emacs 28+, and will be
even more so when Doom later starts byte-compiling its libraries.
There are two issues here.
1. Projectile uses file-remote-p to check for remote (tramp) paths in
its known project list, when it automatically cleans it up on
projectile-mode's activation. This causes tramp.el to be loaded,
which is expensive.
2. file-remote-p relies on an entry in file-name-handler-alist
(autoloaded by tramp.el) to detect remote paths, which causes tramp
to be loaded. However, Doom sets file-name-handler-alist to nil at
startup for a noteable boost in startup performance.
Normally, this is not an issue, as I defer projectile-mode until well
after file-name-handler-alist is restored, but it is trivial for a
user to inadvertantly load it too early (often as part of another
package that depends on it, or by blindly following projectile's
install instructions and calling projectile-mode themselves).
In order to address both of these, I defer projectile's cleanup process
altogether. Another approach I considered was to ensure projectile-mode
wasn't activated until the right time, regardless of when projectile is
loaded, but this may trouble savvier Emacs users who need projectile's
API early during startup, so it needs more consideration.
Fix: #6552
Ref: bbatsov/projectile#1649
Rather than impose a 10-45min compilation step on users, I've disabled
ahead-of-time compilation for deferred compilation. In exchange, it will
eat up some CPU time the first time each uncompiled package is loaded,
but as this happens asynchronously (and are then quietly loaded in the
background), I think this is acceptable.
An --aot switch (or similar) will be added to `doom sync` and `doom
build` in the future, in case folks prefer the old behavior.
In the future, doom-lib (among other things) will be byte-compiled as
part of 'doom sync'. To spare runtime the overhead of checking for these
functions, I've wrapped these in a macro. It also makes their
definitions a tad simpler.
This was removed upstream, and while we haven't bumped use-package for
this to affect us, I may as well remove it early.
Doom will be dropping use-package from core in 3.0, in any case.
Fix: #6699
Ref: 8a3d29e433
This was brought over from `doom-info` in f33d8e7, but one of the
lexical function calls wasn't refactored out.
Ref: a5c80fcb4b/lisp/lib/debug.el (L216-L219)Fix: #6698
Amend: c5e3f4d632
Co-authored-by: ivanbrennan <ivanbrennan@users.noreply.github.com>
If defer-feature! was called with one argument (as is the case in the
:lang common-lisp module), FNS defaulted to an empty list. As a result,
FEATURE was deferred but never re-added to the features list, and after!
blocks were never triggered.
Instead of defaulting to an empty list, fallback to a singleton list
containing just (FEATURE). This aligns with the behavior this macro had
prior to 5b8b04f0c8, which generalized FNS
to support a list of functions rather than just one.
It used to be that after! suppressed macro expansion, but at some point
around 28.1, the elisp interpreter started recognizing the
compiler-macro hint in eval-after-load's definition; implicitly wrapping
quoted forms in a function. Therefore, we can no longer rely on
eval-after-load to hide macros from the byte-compiler. Instead, modules
will need to take care to wrap macro calls in `eval` or similar, on a
case-by-case basis.
This fixes a couple bugs with this macro:
- Nested %-refs (in nested fn!'s) were interpolated as arguments of the
outer-most fn!. E.g. (fn! (fn! %2)) would expand to:
Before this fix:
(lambda (_%1 %2)
(lambda (_%1 %2)
%2))
After this fix:
(lambda ()
(lambda (_%1 %2)
%2))
- Unused arguments were not only listed in the wrong order, they were
off-by-one. E.g.
(fn! %3 %5) expands to (lambda (_%4 _%3 %3 _%1 %5) %3 %5)
This never caused any actual issues, but it was unexpected.
I've also moved the lookup table to `fn!`, and removed unnecessary
entries from it.
startup-redirect-eln-cache adds the new directory and removes
$EMACSDIR/eln-cache from native-comp-eln-load-path, but it's not
available in 28.1, so we'll have to wait until Doom drops 28.1 support
to use it.
doom-etc-dir will be renamed to doom-data-dir, to better reflect its
purpose, and align it with XDG_DATA_HOME (where it will be moved to in
v3, where Doom will begin to obey XDG directory conventions more
closely).
- Deprecates the doom-private-dir variable in favor of doom-user-dir.
- Renames the pseudo category for the user's module: :private -> :user.
- Renames the doom-private-error error type to doom-user-error.
Emacs uses the term "user" to refer to the "things" in user space (e.g.
user-init-file, user-emacs-directory, user-mail-address, xdg-user-dirs,
package-user-dir, etc), and I'd like to be consistent with that. It also
has the nice side-effect of being slightly shorter. I also hope
'doom-user-error' will be less obtuse to beginners than
'doom-private-error'.
featurep! will be renamed modulep! in the future, so it's been
deprecated. They have identical interfaces, and can be replaced without
issue.
featurep! was never quite the right name for this macro. It implied that
it had some connection to featurep, which it doesn't (only that it was
similar in purpose; still, Doom modules are not features). To undo such
implications and be consistent with its namespace (and since we're
heading into a storm of breaking changes with the v3 release anyway),
now was the best opportunity to begin the transition.
To reduce redundancy, remove the maintenance hassle that version
constants would impose later on, and rely on built-in
facilities (featurep) more over global variables or doomisms, these
global constants have been deprecated in favor of Emacs "features":
- EMACS28+ -- replace with (> emacs-major-version 27)
- EMACS29+ -- replace with (> emacs-major-version 28)
- NATIVECOMP -- replace with (featurep 'native-compile)
- MODULES -- replace with (featurep 'dynamic-modules)
(These constants will be formally removed when v3 is released. The IS-*
constants are likely next, but I haven't decided on their substitutes
yet)
I also decided to follow native-compile's example and provide features
for Emacs' system features (since system-configuration-features' docs
outs itself as a poor method to detect features):
- dynamic-modules
- jansson
- native-compile -- this one already exists, but will instead be removed
if it's non-functional; i.e. (native-comp-available-p) returns nil.
These are now detectable using featurep, which is fast and built-in.
Some of our comments/docs can come off as disparaging or snide. They're
glimpses of unfiltered frustration or snarky rubber ducking gone too
far, something I can totally sympathize with, as a scatterbrained
tinkerer, unwittingly made responsible for a lot of work that isn't mine
because of Doom's position as a middleman. But now that Doom has a
veritable userbase, I'd like to hold it to a higher standard.
Light-hearted banter and aired grievances in our source code,
documentation, or community are fine if focused on the problem or the
personal/shared experiences of the community (things that offer value or
amusement to others), but it is never acceptable to attack people or
their efforts. Especially not the very people on whose shoulders Doom
stands.
I sincerely apologize if these have offended you.
Amend: b07614037f
A regression introduced in 1d8c61698b. Doom disables its
file-name-handler-alist optimization if in a daemon session or if debug
mode is active.
Fix: #6657
Amend: 1d8c61698b
If you've moved $EMACSDIR, comp-el-to-eln-filename will throw errors
about missing directories/files, rendering 'doom sync' ineffective and
forcing the user to delete $EMACSDIR and reinstall Doom.
Unsetting file-name-handler-alist around a `load` call prevents any
change to this variable from surviving that file's evaluation (e.g. by
packages loaded therein). Since the user's config files are loaded with
this macro, this affects users' configs, which is unacceptable.
Since this optimization is already done in early-init.el, we can get
away with being more selective here.
See 6f1c0f7cc7 for part 1.
Turns out startup.elc likely exists on most Emacs installations (and,
since it's so integral to Emacs, it likely gets special treatment), so
it was a poor heuristic for this fix. Instead, a more variable target
would be calc-loaddefs.el.
On some systems, only calc-loaddefs.el.gz exists (in which case, we
should turn off the optimization). On others, calc-loaddefs.el
exists (so I'll assume it's safe to leave them on). I won't check for
calc-loaddefs.elc because it doesn't matter; calc.el explicitly
calls (load "calc-loaddefs.el") so it is never loaded.
Of course, you can sidestep the entire issue by building Emacs with
--without-compress-install, but it's not practical for users to
know/want to do that.
Amend: 6f1c0f7cc7
Some installs of Emacs do not come with byte-compiled versions of its
bundled elisp files, so when loading them, Emacs falls back to loading
its *.el.gz files. This would be fine if it were not for a startup
optimization Doom employs, where it sets file-name-handler-alist to
nil (and by doing so, robs Emacs of the ability to read compressed
elisp). This causes "symbol's value as variable is void: \213" errors at
startup.
With this commit, Doom now disables this optimization early if it
suspects this applies to your install. But time will tell if it's early
enough.
Ref: https://mail.gnu.org/archive/html/emacs-devel/2022-08/msg00234.html
Since fd 8.3.0 has low availability across distros (see repology link
below), I don't want to make it Doom's minimum supported version.
Instead, I do a quick version check and adjust accordingly. I'll think
up a more elegant solution after v3.
Ref: https://repology.org/project/fd-find/versionsFix: #6618Fix: #6600Close: #6597
- Rename doom-docs--display-header-h -> doom-docs--display-menu-h to
better represent what it does.
- Add comments to describe the purpose of lib/docs, and TODO annotations
to hint at its future and remind me later.
BREAKING CHANGE: Before, 'doom ci' would load
$GIT_WORKING_TREE/.github/ci.el, to give users/projects an opportunity
to provide project-local configuration for bin/doom (mainly for CI/CD).
Now, this ci.el file is no longer loaded and instead, *all* bin/doom
sessions will walk up the file tree and load the first .doomrc it finds.
This gives bin/doom users a more general place configure all of its
commands, and not just 'doom ci' commands.
Extras:
- Adds .doomrc to auto-mode-alist (so that it starts in
emacs-lisp-mode).
- Adds -o/--file option,
- If -o/--file is passed a dash, print codeowners to stdout,
- Adds --dryrun option,
- Will accept literal string entries in doom-make-codeowners as
standalone lines (useful for comments).
CLIs can now use this for implicit validation for options that take a
file path or - to signal "print to stdout", like so:
(defcli! (doom command) ((outfile ("--out" (file stdout))))
(if (equal outfile "-")
(print! "output")
(with-temp-file outfile
(insert "output"))))
If OUTFILE is not an existing file path or a -, you'll see an this
helpful error:
Error: -o/--file received invalid value "FOO"
Validation errors:
- Must be a dash to signal stdout.
- File does not exist.
See 'doom h[elp] make codeowners' or 'doom make codeowners {-?,--help}' for documentation.
Now you can specify more than one allowed (implicit) for a CLI option:
(defcli! (doom somecommand) ((foo ("--foo" (file int)))))
This will test FOO to ensure it is either an existing file path or an
integer. If neither is true, you'll see this helpful error:
Error: -o/--file received invalid value "FOO"
Validation errors:
- Not an integer.
- Not a valid path to an existing file.
doom-enlist is now a deprecated alias for ensure-list, which is built
into Emacs 28.1+ and is its drop-in replacement. We've already
backported it for 27.x users in doom-lib (in 4bf4978).
Ref: 4bf49785fd
20d5440 introduced a regression where options would be lost when a CLI
session is restarted. :config literate users, for example, would run
'doom sync -u' only for the -u option to be ignored, because it was
discarded after the literate module restarts the session.
Amend: 20d5440023
I've omitted docs/*.org from this merge, as there is still work left to
do there, but I am pushing the module docs early so folks can benefit
from the new docs sooner.
In some edge cases, an early call to doom-log will cause an autoload
error because of one or more of the following is true:
- The autoloads file hasn't been generated or loaded,
- The autoloads file is out of date (especially relevant after
b9933e6),
- doom-cli hasn't loaded lib/files eagerly yet.
To avoid this, and due to how prolific doom-log's uses are, and how
trivial the dependency is, I simply remove the dependency.
Amend: b9933e6637
In the future, should doom-core-dir or {doom-core-dir}/packages.el
change, 'doom upgrade' won't leave the repo in a broken state.
Unfortunately, this cannot retroactively prevent the issue; users will
experience this as soon as they update to b9933e6 or beyond, so users
will have to upgrade manually to overcome it:
cd ~/.emacs.d
git reset --hard 35a89bdfa6
git pull origin master
doom sync -u
Fix: #6598
Amend: b9933e6637
BREAKING CHANGE: This restructures the project in preparation for Doom
to be split into two repos. Users that have reconfigured Doom's CLI
stand a good chance of seeing breakage, especially if they've referred
to any core-* feature, e.g.
(after! core-cli-ci ...)
To fix it, simply s/core-/doom-/, i.e.
(after! doom-cli-ci ...)
What this commit specifically changes is:
- Renames all core features from core-* to doom-*
- Moves core/core-* -> lisp/doom-*
- Moves core/autoloads/* -> lisp/lib/*
- Moves core/templates -> templates/
Ref: #4273
