* Fix Git HTTP smart host detection
regression test test/issues/el-get-issue-1920.el revealed that Git
HTTP smart host detection is broken. Not all hosts support the HEAD
HTTP request method. For example, github.com for HEAD
(let ((url-request-method "HEAD"))
(url-retrieve-synchronously
"https://github.com/dimitri/el-get.git/info/refs\?service\=git-upload-pack"))
responds
HTTP/1.1 405 Method Not Allowed
Server: GitHub Babel 2.0
Content-Type: text/plain
Content-Security-Policy: default-src 'none'; sandbox
Content-Length: 0
X-Frame-Options: DENY
X-GitHub-Request-Id: C1DA:12E32:2B6C9D7:302B1C2:61C9A4E8
while for GET
(let ((url-request-method "GET"))
(url-retrieve-synchronously
"https://github.com/dimitri/el-get.git/info/refs\?service\=git-upload-pack"))
responds
HTTP/1.1 200 OK
Server: GitHub Babel 2.0
Content-Type: application/x-git-upload-pack-advertisement
Content-Security-Policy: default-src 'none'; sandbox
Transfer-Encoding: chunked
expires: Fri, 01 Jan 1980 00:00:00 GMT
pragma: no-cache
Cache-Control: no-cache, max-age=0, must-revalidate
Vary: Accept-Encoding
X-Frame-Options: DENY
X-GitHub-Request-Id: C22C:5E15:3923777:3D822AC:61CA332A
Other hosts like git.sr.ht do support HEAD, of course.
Furthermore, the HTTP status code wasn't checked, that's why hosts
like github.com would be classified as "dumb" hosts.
This commit checks the HTTP status code, and if the status is not 200
or 304 for the HEAD HTTP request method, it tries GET. HEAD is tried
first, because GET might be more expensive for big repositories.
The regression test is adapted as well.
* Fix byte compiler warnings
Since Emacs 27 the package cl is deprecated, the replacement is
cl-lib, which is available since Emacs 24.3.
This patch replaces cl by cl-lib and drops support for Emacs versions
less than 24.3. Dropping older Emacsen is required, because cl-lib is
a builtin starting from version 24.3 and doesn't need an extra package
from ELPA.
Testcases for past issues still contain cl. Most of them seem to be
broken and need further investigation.
This patch is tested with test/run-ert.sh, which outputs:
Ran 10 tests, 10 results as expected, 0 unexpected (2021-01-30 13:24:54+0100, 0.672122 sec)
1 expected failures
and manually by daily usage for a month now.
If the "apt-get install" and "apt-get remove" are not run synchronously,
the later will prevent the former to run, because it is not possible to run
two dpkg processes simultaneously on a Debian-based system.
When a package is not installed in the system, the dpkg-query command exits
with an error. This will then make the function
el-get-dpkg-package-installed-p fails, which interrupts the installation of
the package, making the apt-get method useless. This commits avoids the
problem by wrapping the call to process-lines into a condition-case
statement.
In Emacs 26, copy-directory copies the directory itself, based on
whether NEWNAME is directory name (ends with a "/"). Earlier Emacs
versions actually checked whether NEWNAME was existing directory, so
we happened to get the right behaviour most of the time.
Pass t for COPY-CONTENTS to ensure the correct behaviour all of the
time.
* methods/el-get-elpa.el (package-desc-archive): Define it for old Emacs
versions that don't provide it.
(el-get-elpa-build-local-recipes): Use that instead of hard-coding
offset into package-desc array.
* el-get-methods.el (el-get-insecure-check): Also consider URLs
satisfying `file-name-absolute-p' to be secure. `package-archives' uses
absolute file names *without* file:// prefix, so we have allow this too.
* methods/el-get-elpa.el (el-get-elpa-package-id):
(el-get-elpa-package-archive-base): New compat functions.
* methods/el-get-elpa.el (el-get-elpa-install): Call
`el-get-insecure-check' after ensuring `package-archive-contents' is
initialized.
github method, as a derived method, should build a final repository URL
and let parent's respective methods do the actual work.
Register new 'el-get-github-pull function as update method.
Make both methods, 'el-get-github-pull and 'el-get-github-clone,
implement the same pattern:
- Delegate check if URL is a secure one to the parent method
- Ignore :url from package's recipe passed by 'el-get-do-update as it
makes no sense
- Unconditionally build package repository URL with 'el-get-github-url
- Call respective git method
in name of buffer, rather than el-get package name. Since we write what
looks like the command being executed, "apt-get install FOO", it's
confusing if FOO isn't the actual package being passed to apt-get.
Fixes#2358.
This avoids asking for root priviledges if it's not needed.
* methods/el-get-apt-get.el (el-get-dpkg-package-installed-p): Renamed
from el-get-dpkg-package-status, don't use shell.
(el-get-apt-get-install-if-needed): New function.
(el-get-register-method :apt-get): Use it as :install method.
Escape the value of :description properly generating recipes.
Make auto generation of recipes from package.el descriptors quieter and
faster (forego nice indentation).
Both the http-tar and http-zip methods are modified to manually
verify the checksum before handling the archive. This is a
security precaution and also prevents unexpected consequences from
attempting to work with a corrupted archive file.
The checksum verification code is factored out of el-get-post-install
so that the tar and zip methods can verify using the same code as
other methods.
The auto-generated is simple enough that it's pretty readable without
indentation, and on Emacs 24.4 the indent-region function prints a
message.
Also, since we didn't change to elisp mode, the indentation wasn't doing
anything until now anyway.
* methods/el-get-emacswiki.el (el-get-emacswiki-retrieve-package-list):
Throw error if we didn't get HTTP 2xx.
* el-get-install.el: Show warning if
`el-get-emacswiki-build-local-recipes' fails.
This prevents the sha1 function from attempting and then failing to
encode the buffer contents in the wrong encoding.
* methods/el-get-http.el (el-get-http-compute-checksum): Call `sha1' on
result of `buffer-string', not `current-buffer'.
Using just message makes it too easy to overlook. The warning only
triggers on installation (not update or init) so it shouldn't be too
annoying.
* methods/el-get-apt-get.el (el-get-dpkg-symlink): use lwarn instead of
message.
If the apt-get package doesn't contain any elisp files symlinking to the
non-existant /usr/share/emacs/site-lisp/<package> directory will leave
us with a broken link that will raise an error when we attempt to look
for autoloads, clean stale elc files and the like. In this case, simply
create an empty directory instead of a link.
* methods/el-get-apt-get.el (el-get-dpkg-symlink): call `make-directory'
instead of symlink when `debdir' doesn't exist.
* methods/el-get-elpa.el (el-get-elpa-install-1-package): rename to
el-get-elpa-install-package, take another argument have-deps-p. Only
call package-download-transaction directly if have-deps-p.
(el-get-elpa-install, el-get-elpa-update): el-get-elpa-install{-1
=>}-package name change and new arg.
* methods/el-get-elpa.el (el-get-elpa-install-1-package): new wrapping
function for package-download-transaction.
(el-get-elpa-install, el-get-elpa-update): use it.
Since el-get also downloads dependencies we end up installing things
twice.
* methods/el-get-elpa.el (el-get-elpa-install): call
package-download-transaction instead of package-install.
The current algorithm is
1) If the protocol used is not http (file, ssh, git) clone is supported
2) Otherwise check if repo belongs to know smart host, if so assume
shallow clone is supported
3) If none of the above work, make a HEAD request and parse response
headers to determine the host is smart explained (here)[http://stackoverflow.com/questions/9270488/]
instead of shell-command which is sensitive to the user's choice of
shell.
* el-get-elpa.el (el-get-elpa-symlink-package): use make-symbolic-link
instead of shell-command "ln -s %s %s".
In most cases, we assume any connection is insecure unless the URL
starts with "https://", "$USERNAME@", or "ssh". There are a few
exceptions: I'm assuming all Emacswiki packages are insecure, and I
don't think we can know whether packages installed via Google Go are
secure or not.
When downloading a .gz file, if we actually write-file to that filename
the compressed data would be compressed a second time thanks to
auto-compress-mode. Instead, write to .part file first and then rename
to the destination, as it was before
92a39c84fe.
