The prior implementation of 'parse' was, to put it mildly, very accepting of
invalid input.  This made it very difficult to review its security properties,
which ended up relying entirely on the fact that it still didn't allow "/" in
directory entry names and that O_CREAT|O_EXCL implicitly causes symlinks to
not be followed in the final component, and mkdir and symlink never follow
symlinks in the final component, and all of the above throw exceptions when
they fail.

Some examples of what was allowed prior:

1. Empty archives:
   "nix-archive-1" "(" ")"
2. Archives with only a type:
   "nix-archive-1" "(" "type" "regular" ")"
3. Archives of regular files with multiple contents, which would be
   concatenated (e.g. this has contents "foobar"):
   "nix-archive-1" "(" "type" "regular" "contents" "foo" "contents" "bar" ")"
4. Archives of directories with multiple entries with the same name, and that
   aren't sorted:
   "nix-archive-1" "(" "type" "directory"
                       "entry" "(" "name" "c" "node" "(" ")" ")"
                       "entry" "(" "name" "b" "node" "(" ")" ")"
                       "entry" "(" "name" "b" "node" "(" ")" ")"
                       "entry" "(" "name" "a" "node" "(" ")" ")" ")"
5. Archives of directories with empty entries:
   "nix-archive-1" "(" "type" "directory" "entry" "(" ")" ")"
6. Archives of directories with entries with no name:
   "nix-archive-1" "(" "type" "directory" "entry" "(" "node" "(" ")" ")" ")"
7. Archives of directories with entries with no node:
   "nix-archive-1" "(" "type" "directory" "entry" "(" "name" "a" ")" ")"
7. Archives of directories with entries with multiple names:
   "nix-archive-1" "(" "type" "directory" "entry" "(" "name" "a"
                                                      "node" "(" ")"
                                                      "name" "b" ")" ")"
8. Archives of directories with entries with multiple nodes:
   "nix-archive-1" "(" "type" "directory" "entry" "(" "node" "(" ")"
                                                      "node" "(" ")" ")" ")"
9. Archives of directories with entries with multiple names and multiple
   nodes:
   "nix-archive-1" "(" "type" "directory"
                       "entry" "(" "name" "a" "node" "(" ")"
                                   "name" "b" "node" "(" ")" ")" ")"

Such permissiveness rather defeats the "normalized" part of "Normalized
ARchive".

Additionally, 'parse' previously used recursion with no explicit depth limit.
In practice Linux would still limit the depth to around 2048, since filenames
longer than 4096 bytes yield ENAMETOOLONG, but no such restriction exists on
Hurd.

The permissiveness issue is resolved by rewriting 'parse', and the unbounded
recursion issue is resolved by adding a nestLimit parameter so that no more
than DIRECTORY_NESTING_LIMIT (currently 256) levels of recursion are used.  An
alternate implementation that doesn't use recursion, 'parse_unbounded', is
included in case it is determined that the nesting limit should be removed,
but currently only 'parse' is used.

* nix/libutil/archive.cc (parse): rewrite to add nestLimit parameter.
  (parseDump): pass DIRECTORY_NESTING_LIMIT as nestLimit argument to 'parse'.
  (parse_unbounded): new function.

Change-Id: Ia3c32cc645b609f6ae7f3bcafc491ec0073cbb51
Signed-off-by: Ludovic Courtès <ludo@gnu.org>

* nix/libutil/hash.hh (isHash32): new function.
* nix/libutil/hash.hh (isHash32): provide implementation.

Signed-off-by: Ludovic Courtès <ludo@gnu.org>

Recognizing base32 characters and/or parsing them into byte sequences is done
in several places, so it makes sense to provide a common lookup table.

* nix/libutil/hash.hh (base32Values, base32ValuesInitialized): new variables.
  (initializeBase32Values, getBase32Value): new functions.
* nix/libutil/hash.cc (base32Values, base32ValuesInitialized): provide
  definition for variables.
  (initializeBase32Values, getBase32Value): provide function implementations.
  (parseHash32): use base32Values.
* nix/libstore/references.cc (search): use base32Values.

Signed-off-by: Ludovic Courtès <ludo@gnu.org>

Fixes a regression introduced in 7173c2c0ca.  Additional discussion at
https://codeberg.org/guix/guix/pulls/5977.

* nix/libutil/util.cc (_deletePathAt): chmod directory and retry open when it
  fails with EACCES.  Do this using an O_PATH file descriptor referenced via
  /proc/self/fd whenever possible to avoid it being replaced by a
  non-directory immediately before being chmod'ed.
* nix/libutil/util.hh (deletePath): document TOCTTOU race on non-linux systems
  where hardlinks aren't protected.
* tests/derivations.scm ("unreadable directories in build tree can be
  removed"): new test.

Fixes: guix/guix#5891
Reported-by: Liliana Marie Prikler <liliana.prikler@gmail.com>
Change-Id: I749127fe5254ebabc8387a2f0ef47e3c116bfcc5
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
Merges: #6460

Change-Id: Id73c0c74600a3da3373b3a37236b505af65bfe31

* nix/libstore/globals.hh,
nix/libstore/store-api.hh,
nix/libutil/util.hh: Fix misspellings in comments.

Change-Id: Id77ea2bde1c2582d1a7bec4ed256ea900998c4b8

This is a followup to 3af52f845fe2ceb448416ac7b9f48925673c594e and
6f1448ef89c8ad29e2a479099531fd3a87701e46.

* nix/libstore/local-store.cc (LocalStore::queryValidPathId): Change %1% to {}.
* nix/libstore/optimise-store.cc (LocalStore::optimisePath_): Likewise.
* nix/nix-daemon/guix-daemon.cc (open_unix_domain_socket): Likewise.
(main): Likewise.
* nix/libutil/affinity.cc (restoreAffinity): Remove useless %1%.

Change-Id: I3a7cd79cd69ab58f14118662f1dcf2fef067a6b7

Avoid separate declarations and definitions for so-called 'global' objects.

* nix/libstore/derivations.{cc, hh} (drvHashes): Use inline instead of separate
declaration and definition.
* nix/libstore/globals.{cc, hh} (settings, nixVersion): Same.
* nix/libstore/local-store.hh (drvsLogDir, deduplicationMinSize): Same.
* nix/libstore/optimise-store.cc (deduplicationMinSize): Same.
* nix/libstore/store-api.{cc, hh} (store): Same.
* nix/libutil/archive.{cc, hh} (defaultPathFilter): Same.
* nix/libutil/hash.{cc, hh} (base32Chars): Same and modify header files.
* nix/libutil/util.{cc, hh} (logType, verbosity, _writeToStderr,
_isInterrupted): Same.
* nix/local.mk: Modified according to the rename of shared.hh.
* nix/nix-daemon/guix-daemon.cc (blockInt, argvSaved, run): Same and
modify header files.
* nix/nix-daemon/nix-daemon.cc: Modify header files.
* nix/nix-daemon/shared.hh: Renamed to nix-daemon.hh

Signed-off-by: Ludovic Courtès <ludo@gnu.org>

* nix/libstore/build.cc (DerivationGoal::tryBuildHook): Use starts_with instead
of string()
* nix/libstore/builtins.cc (lookupBuiltinBuilder): Same.
* nix/libstore/builtins.hh (isBuiltin): Same and fix indentation of the file.
* nix/libstore/derivations.cc (DerivationOutput::parseHashInfo, isDerivation):
Same and clean header files.
* nix/libstore/gc.cc (addPermRoot, LocalStore::isActiveTempFile): Same.
* nix/libstore/globals.cc: Same.
* nix/libstore/local-store.cc: Same.
* nix/libstore/misc.cc: Same.
* nix/libstore/store-api.cc (checkStoreName): Same.
* nix/libutil/affinity.cc: Same.
* nix/libutil/archive.cc: Same.
* nix/libutil/spawn.cc: Same.
* nix/libutil/util.{cc, hh} (hasSuffix): Removed.

Signed-off-by: Ludovic Courtès <ludo@gnu.org>

* nix/boost: This directory and all files inside it are removed.
* nix/libstore/build.cc (Goal::trace): Use ‘std::string’ instead of ‘const format &’.
(DerivationGoal::startBuilder, ...): Use ‘std::format’ or ‘std::vformat’ instead of ‘boost::format’.
* nix/libstore/builtins.cc (builtinDownload): Same.
* nix/libstore/derivations.cc (DerivationOutput::parseHashInfo, ...): Same.
* nix/libstore/gc.cc (LocalStore::openGCLock, ...): Same.
* nix/libstore/globals.cc (Settings::_get): Same.
* nix/libstore/local-store.cc: (checkStoreNotSymlink, ...): Same.
* nix/libstore/misc.cc (dfsVisit, showBytes): Same
* nix/libstore/optimise-store.cc (makeWritable, ...): Same.
* nix/libstore/pathlocks.cc (openLockFile, ...): Same.
* nix/libstore/references.cc (search, scanForReferences): Same.
* nix/libstore/sqlite.hh (throwSQLiteError): Use ‘std::string’ instead of ‘const format &’.
* nix/libstore/sqlite.cc (throwSQLiteError): Use ‘std::string’ instead of ‘const format &’.
* nix/libstore/store-api.cc (assertStorePath, ...): Use ‘std::format’ instead of ‘boost::format’.
* nix/libutil/affinity.cc (setAffinityTo): Same.
* nix/libutil/archive.cc (dumpContents, ...): Same.
* nix/libutil/hash.cc (parseHash, parseHash32, parseHash16or32, hashFile): Same.
* nix/libutil/hash.hh (parseHash, parseHash32, parseHash16or32, isHash): Same.
* nix/libutil/serialise.cc : Add ‘<cassert>’ header file.
* nix/libutil/spawn.cc (addPhaseAfter, ...): Use ‘std::format’ instead of ‘boost::format’.
* nix/libutil/types.hh (FormatOrString): Removed.
(BaseError, BaseError::addPrefix, SysError, MakeError):
Use ‘std::string or std::string_view’ instead of ‘FormatOrString’.
* nix/libutil/util.hh (Nest::open, printMsg_, warnOnce, expect): Same.
* nix/libutil/util.cc (BaseError::BaseError, ...): Same.
(writeToStderr, _interrupted): Use std::uncaught_exceptions() instead of std::uncaught_exception()
* nix/nix-daemon/nix-daemon.cc (performOp, ...): Same.
* nix/nix-daemon/guix-daemon.cc (string_to_bool, ...): Same.
* nix/local.mk: Remove ‘libformat.a’ from ‘noinst_LIBRARIES’,
remove ‘libformat_a_SOURCES’ and ‘libformat_headers’,
remove ‘libformat_a_CPPFLAGS’ from ‘libutil_a_CPPFLAGS’ and ‘guix_daemon_LDADD’,
update ‘AM_CXXFLAGS’ to ‘-std=c++20’.

Signed-off-by: Ludovic Courtès <ludo@gnu.org>