path: root/fs
Commit message (Collapse)AuthorAgeFilesLines
* fs/cpio: add zstd as compression optionGravatar José Luis Salvador Rufo2021-01-162-0/+10
| | | | | | | | U-Boot mkimage zstd is available since v2020.10: https://gitlab.denx.de/u-boot/u-boot/-/commit/26073f9ed3ab0aaf3c2a2b433fecb30a95a067d6 Signed-off-by: José Luis Salvador Rufo <salvador.joseluis@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* fs/jffs2: copy xattrsGravatar Antoine Tenart2020-10-081-1/+1
| | | | | | | | | This patch sets the --with-xattr option when generating an image with mkfs.jffs2, so that SELinux security contexts and other extended attributes will be kept and available in jffs2 images. Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* fs/common.mk: move BR2_ROOTFS_POST_FAKEROOT_SCRIPT after pre cmd hooksGravatar Charles Hardin2020-09-271-2/+2
| | | | | | | | | | | | | | | | | | | The fakeroot script does not appear to be used in any of the checked in defconfig targets, but it seems that most often the post fakeroot script should be done after all the packages rules have been applied instead of before. Given that a change in systemd moved the SYSTEMD_PRESET_ALL hook to a ROOTFS_PRE_CMD_HOOKS, there was no way to use a FAKEROOT script to disable a service or fixup a systemd configuration. The systemd move makes sense, and this just tries to preserve the same ability to fixup a rootfs after all the cmd hooks are processed. Refer to commit 65b63785a642bbc565924edf365b5d7c080f961c for the change that instigated this reordering. Signed-off-by: Charles Hardin <ckhardin@gmail.com> Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
* fs/common.mk: move down ROOTFS_REPRODUCIBLE for consistencyGravatar Antoine Tenart2020-09-041-6/+6
| | | | | | | | This patch is cosmetic and moves down ROOTFS_REPRODUCIBLE for consistency. Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* fs/common.mk: set SELinux file security contextsGravatar Antoine Tenart2020-09-041-0/+11
| | | | | | | | | | | | | | | | | | | | | Set the SELinux file security contexts using setfiles when generating root filesystem images. Without such security contexts created at build time, they need to be setup at first boot by running the restorecon utility on the target. This has two drawbacks: - You have to special case the first boot, which cannot be done in enforcing mode, and will have to run restorecon, then reboot. - You cannot support read-only filesystems. By setting up the security contexts at build time, we can have a filesystem image that is immediately ready to boot an SELinux system in enforcing mode, including if the root filesystem is read-only. Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* fs/cpio/init: unbreak ttyname_r() on glibc after dropping /dev/console execGravatar Peter Korsgaard2020-08-291-0/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit 98a6f1fc02e41 (fs/cpio: make initramfs init script survive 'console=' kernel argument) dropped the explicit /dev/console execs for fd 0,1,2, as they fail when booted with console= and aren't really needed as the kernel will setup fd 0,1,2 from /dev/console before executing the initramfs anyway. Not doing this unfortunately confuses glibc's ttyname_r(3) implementation (used by E.G. busybox/coreutils 'tty'), causing it to fail with ENOENT as it does a fstat on fd 0 and tries to match up st_ino / st_dev against the entries in /dev (since glibc 2.26): commit 15e9a4f378c8607c2ae1aa465436af4321db0e23 Author: Christian Brauner <christian.brauner@canonical.com> Date: Fri Jan 27 15:59:59 2017 +0100 linux ttyname and ttyname_r: do not return wrong results If a link (say /proc/self/fd/0) pointing to a device, say /dev/pts/2, in a parent mount namespace is passed to ttyname, and a /dev/pts/2 exists (in a different devpts) in the current namespace, then it returns /dev/pts/2. But /dev/pts/2 is NOT the current tty, it is a different file and device. Detect this case and return ENODEV. Userspace can choose to take this as a hint that the fd points to a tty device but to act on the fd rather than the link. Signed-off-by: Serge Hallyn <serge@hallyn.com> Signed-off-by: Christian Brauner <christian.brauner@ubuntu.com> The reason it fails is that we manually mount devtmpfs on /dev in /init, so the /dev/console used by the kernel (in rootfs) is not the same file as /dev/console at runtime (in devtmpfs). Notice: Once logged in, tty does work correctly. Presumably login reopens stdin/stdout/stderr. To fix this, re-add the exec of /dev/console for fd 0,1,2, but only do so if possible. Because of the above mentioned shell behaviour (specified by POSIX [0]), perform this check in a subshell. [0] https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_20_01 Signed-off-by: Peter Korsgaard <peter@korsgaard.com> Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
* fs/cpio: generate reproducible archivesGravatar Yurii Monakov2020-06-291-1/+5
| | | | | | | | | | | | | | | | | | The output of 'find' depends on the directory entries, and is not ordered. As a consequence, the cpio archive is not reproducible. Fix that by sorting the output of find. Use the 'C' locale to enforce reproducibility that does not depend on the locale. The command line is now pretty long, so we wrap it. Signed-off-by: Yurii Monakov <monakov.y@gmail.com> [yann.morin.1998@free.fr: - use LC_ALL=C when sorting - wrap long line - reword commit log ] Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
* fs: squashfs: Add config option to pass nopad parameter to mksquashfsGravatar Christoph Muellner2020-06-172-0/+12
| | | | | | | | | Add an option to enable/disable padding to a multiple of 4k. Padding is the default as it also was the past default behaviour. Signed-off-by: Christoph Muellner <christoph.muellner@theobroma-systems.com> Signed-off-by: Heiko Stuebner <heiko.stuebner@theobroma-systems.com> Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
* fs/ubi: expose BINARIES_DIR to ubinize.cfgGravatar Alexandre Belloni2020-06-062-1/+6
| | | | | | | | | | | | | | | | It is often necessary to refer to other images that are present in BINARIES_DIR from a custom ubinize configuration e.g. to include the kernel in a UBI volume. As we do for BR2_ROOTFS_UBIFS_PATH, replace BINARIES_DIR when copying the file. Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> [yann.morin.1998@free.fr: - add the blurb in the help text - rewrap commit log ] Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
* fs/cpio: make initramfs init script survive 'console=' kernel argumentGravatar Timo Ketola2020-04-251-3/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When booting with 'console=<empty>' in the kernel command line (as e.g. U-Boot does with silent flags in effect), opening /dev/console fails. As per POSIX [0], when iany redirection fails, the shell running exec shal exit in error. So, when 'console=<empty>' is specified. /dev/console can't be opened, and the redirection fails, and /init is killed. That behaviour was fixed on the kernel side with commit 2bd3a997befc2 (Open /dev/console from rootfs), present since 2.6.34, released in May 2010, so any [dr]ecent kernel will have that fix. Furthermore, busybox will fix things up anyway (in bb_sanitize_stdio()), falling back to opening /dev/null if no console is availble. systemd does a similar thing (in make_console_stdio()), and sysvinit again has a similar approach (in console_init()). The archealogy search turned up those relevant commits: 2011-08-04 10a130f91e initramfs/init: make sure that 0, 1, 2 fds are available introduces the three exec redirections in initramfs 2011-09-06 3fac21ef8d cpio: fix boot with dynamic /dev introduces the three exec redirections in cpio 2011-09-06 13a3afc536 fs/initramfs: refactor with fs/cpio dropped the initramfs tweaks to reuse the cpio ones 2012-11-04 e1ebae700a fs/common: Create initial console device introduces the /dev/console char,5,1 pseudo device creation in cpio 2018-03-31 dec061adce fs/cpio: don't extend packages' permissions table switched from the permission-table to a manual mknod to create /dev/console The redirections were added before we could guarantee there was a /dev/console in the rootfs. We're now guaranteed to have /dev/console in an initramfs, and any recent kernel will automatically open /dev/console before spawning /init. The three redirections are useless now, and cause harm under certain conditions. Drop them. [0] https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_20_01 Signed-off-by: Timo Ketola <timo.ketola@exertus.fi> Cc: Peter Korsgaard <peter@korsgaard.com> [yann.morin.1998@free.fr: - extend commit log with the analysis done with Peter ] Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
* fs/ext2: pass a default string to filesystem labelGravatar Fabio Estevam2020-04-151-0/+1
| | | | | | | | | | | | Currently the volume-label for the root filesystem partition is a string wit the following pattern: ad09a287-46a9-4790-ba97-fbbb549e5e96. Specify the volume-label as "rootfs" to make it easier to identify it. Suggested-by: Arnout Vandecappelle <arnout@mind.be> Signed-off-by: Fabio Estevam <festevam@gmail.com> Tested-by: Pierre-Jean Texier <pjtexier@koncepto.io> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* fs/erofs: add support for creating EROFS rootfs imageGravatar Gao Xiang2020-03-193-0/+32
| | | | | | | | | | | | This patch makes possible to create rootfs image using EROFS filesystem. Signed-off-by: Gao Xiang <hsiangkao@aol.com> [yann.morin.1998@free.fr: - simplify help text of filesystem entry - drop the compression choice, keep the single boolean ] Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
* fs/initramfs: fix show-infoGravatar Yann E. MORIN2020-03-181-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The initramfs is not a reall filesystem, so it does not use the $(rootfs) infrastructure. As a consequence, the usual rootfs-related variables are not set, especially the name, type, and dependencies of the (non-)filesystem. Yet, it is present in the list of rootfs to build, and thus we end up including it in the output of show-info. But the missing variables yield an incorrect json: "": { "type": "", "virtual": false, "version": "", "licenses": "", "dl_dir": "", "install_target": , "install_staging": , "install_images": , "downloads": [ ], "dependencies": [ ], "reverse_dependencies": [ ] }, First, the object key is empty; second, the install_target, install_staging, and install_images values are empty, which is not valid (if they were null, that be OK though). Third, this is clearly the layout of a 'package' entry, not that of a 'rootfs' entry. An option to fix that would be to actually make use of the rootfs infra. However, that would mean doing a lot of work for nothing (there is actually nothing to do, yet the infra would still do a lot of preparatory and clean up work). The alternative is pretty simple: declare and set the variables as if it were a real filesystem, so that show-info can filter it to the proper layout and can spit out appropriate content (even if fake). The third option would be to teach show-info (and its internal implementation, the macro json-info) to ignore specific cases, like no-name items, or replace empty values with null, or whatnots. This again would be quite a lot of work for a single occurence. So we go for the simple faked variables. We add linux as a dependency, so that the graph-depends also properly represent the dependency chain, which ends up with something liKe: ALL | v rootfs-initramfs | | v v linux rootfs-cpio which is pretty fitting in the end. Reported-by: Thomas De Schampheleire <patrickdepinguin@gmail.com> Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr> Tested-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* fs: don't use := when not neededGravatar Yann E. MORIN2019-10-276-7/+7
| | | | | Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
* fs/common.mk: enable multithreaded xz compressionGravatar James Hilliard2019-08-031-0/+3
| | | | | | | | | | | | | | | xz help indicates only 1 thread is used unless we set threads: -T, --threads=NUM use at most NUM threads; the default is 1; set to 0 to use as many threads as there are processor cores Since this splits the file into blocks, the result will be not bit-for-bit identical to single-threaded compression. Therefore, don't enable this in BR2_REPRODUCIBLE builds. Signed-off-by: James Hilliard <james.hilliard1@gmail.com> Reviewed-by: Matthew Weber <matthew.weber@rockwellcollins.com> [Arnout: append the option instead of repeating the entire command] Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
* fs/cpio: fix typo in commentGravatar Thomas Petazzoni2019-07-171-1/+1
| | | | | | --reproducible is --reproducible, not --repoducible. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* fs/cpio: make cpio rootfs reproducibleGravatar Atharva Lele2019-07-171-1/+8
| | | | | | | | | | | | | | | | | | Pass the recommended argument in the CPIO manual to make cpio archives reproducible. Reference: https://www.gnu.org/software/cpio/manual/cpio.html#Copy_002dpass-mode Pre-patch diffoscope output: https://gitlab.com/snippets/1874745 Post-patch: https://gitlab.com/snippets/1874746 We can see that post-patch, the archive related differences are removed. The differences are arising from utils/bin/getconf. This will have to be investigated further. However, that is unrelated to cpio. Signed-off-by: Atharva Lele <itsatharva@gmail.com> Reviewed-by: Yann E. MORIN <yann.morin.1998@free.fr> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* fs/common.mk: do not store original names and timestamps when creating ↵Gravatar Atharva Lele2019-07-171-1/+1
| | | | | | | | | | | | | | gzipped rootfs Using the GZIP environment variable to pass gzip options is deprecated, and therefore we are going to remove the "GZIP = -n" definition from the main Buildroot Makefile. In preparation for this, we explicitly add the -n argument to the gzip call in fs/common.mk to ensure reproducibility. Signed-off-by: Atharva Lele <itsatharva@gmail.com> Acked-by: Yann E. MORIN <yann.morin.1998@free.fr> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* fs/tar: explicitly set extended header values to ensure binary reproducibilityGravatar Atharva Lele2019-07-141-0/+3
| | | | | | | | | | | | | | | | | | Since we use --xattrs-include='*' to include all extended attributes, tar creates a PAX formatted archive. The archive metadata captures atime and ctime of files. To fix this, GNU recommends that we pass this added argument to tar to create binary reproducible packages. Setting of mtime is handled in fs/common.mk using touch on all files. Diffoscope output pre-change: https://gitlab.com/snippets/1871111 Diffoscope output after change is blank i.e. binary reproducibile rootfs is created. GNU Recommendation: https://www.gnu.org/software/tar/manual/tar.html#SEC147 Signed-off-by: Atharva Lele <itsatharva@gmail.com> Reviewed-by: Matt Weber <matthew.weber@rockwellcollins.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* core: remove show-dependency-treeGravatar Yann E. MORIN2019-05-071-8/+0
| | | | | | | | | | | | | | | | | | show-dependency-tree was introduced in this release cycle, as a way to quickly and easily provide the dependency tree to graph-depends. show-dependency-tree is no longer used, now that graph-depends has been switched over to using the more versatile show-info. Beside, show-dependency-tree has never been part of a release. Drop it. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com> Cc: Arnout Vandecappelle <arnout@mind.be> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* core: add per-package and per-filesystem show-infoGravatar Yann E. MORIN2019-04-151-1/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Sometimes, it is need to quickly get the metadata of a subset of packages, without resorting to a full-blown JSON query. Introduce a new per-package (and per-filesystem) foo-show-info rule, that otputs a per-entity valid JSON blob. Note that calling it for multiple packages and.or filesystems at once will not generate a valid JSON blob, as there would be no separator between the JSON elements: $ make {foo,bar}-show-info { "foo": { foo stuff } } { "bar": { bar stuff } } However, jq is able to absorb this, with its slurping ability, which generates an array (ellipsed and manualy reformated for readability): $ make {foo,bar}-show-info |jq -s . - [ { "foo": { foo stuff } }, { "bar": { bar stuff } } ] Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com> Cc: Arnout Vandecappelle <arnout@mind.be> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
* fs: add all recursive dependencies to packages listGravatar Yann E. MORIN2019-04-151-1/+1
| | | | | | | | | | | | | | Currently, only first-level dependencies of a filesystem are added to the global list of packages, thus missing all recursive dependencies. Use the newly introduced recursive variable instead, which already contains the rootfs-common dependencies too. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
* fs: introduce variable with all recursive dependenciesGravatar Yann E. MORIN2019-04-151-0/+24
| | | | | | | | | | | | | This allows getting all the recursive dependencies of filesystems, ike we have for packages, and allows us to treat both in a similar fashion. Reported-by: Arnout Vandecappelle <arnout@mind.be> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
* fs: introduce variables with name and typeGravatar Yann E. MORIN2019-04-151-0/+4
| | | | | | | | | | | | This makes the filesystems resemble packages yet a bit more, and will allow sorting "items" on their type and names, when indexed from the upper-case names. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
* fs/common.mk: disable real chown calls in fakerootGravatar Peter Korsgaard2019-04-061-1/+1
| | | | | | | | | | | | | | | | | | | fakeroot by default forwards {f,l,}chown calls to libc and ignores permission issues, which may cause issues when building in restricted environments like user namespaces as set up with bubblewrap where a chown call with a uid/gid not mapped in the user namespace instead returns EINVAL. This error is not masked by fakeroot and returned to the caller, causing failures. There is no real reason to really perform the *chown calls in the context of Buildroot (as the calls will likely just fail and files are not accessed outside the fakeroot environment any way). This forwarding can be disabled by setting the FAKEROOTDONTTRYCHOWN environment variable, so set it when fakeroot is executed. Reported-by: Esben Nielsen <nielsen.esben@gmail.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* core: add make-based full-dependency listGravatar Yann E. MORIN2019-03-171-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, when we need to build the full dependency graph, we call make to show the list of packages (make show-targets), and then call it again and again iteratively while it returns new packages. Since calling make will parse the whole set of our Makefiles, this takes quite a bit of time (~4s each here), and the total can get pretty long. However, make being make, already builds the whole dependency tree information, so we can just ask for it. Add a new top-level rule 'show-dependency-tree' that displays the whole set of dependencies for all packages. For each package, its name, type and version is displayed, then all the direct, first-level dependencies are dumped. We choose a format that is not unlike the dot-graph format, because it is both easy to read as a human, and easy to parse as a machine: foo: target 1.2.3 foo -> bar host-meh bar: target virtual bar -> buz buz: target 2.3.4 buz -> host-meh: host virtual host-meh -> host-bleark host-bleark: host 3.4.5 host-bleark -> rootfs-meh: host rootfs-meh -> host-bleark To be noted: rootfs are currently reported as if they were 'host' packages, to stay aligned with how graph-depends currently treats them. Ideally, graph-depends could be enhanced to recognise them separately, but that is another story. For just plain defconfig, which is about the smallest config we can have with an internal toolchain, we already have a seven-fold improvement (with the graph-depends rule modified to not run the pdf generation, to be able to just compare the tree generation): $ time make graph-depends real 0m27.344s $ time make show-dependency-tree real 0m3.848s >From defconfig, C++, wchar, locales, ssp, and allyespackageconfig, tweaked for even more packages (qt5 not qt4, luajit to avoid multi providers, etc...), the timings are (graph-depends still modified to not generate the pdf): $ time make graph-depends real 1m56.459s $ time make show-dependency-tree real 0m5.748s There. I don't think those numbers need any explanation whatsoever; they do speak on their own. OK, for maths sake, the ratio is about twenty-fold. So, "yeah", I guess... ;-) Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* fs/common.mk: make sure that static devices from packages are createdGravatar Thomas Petazzoni2018-12-041-1/+1
| | | | | | | | | | | | | | | | | | | The static devices defined by packages are currently added to the full device table when two conditions are met: (1) ROOTFS_DEVICE_TABLES is non-empty (2) BR2_ROOTFS_DEVICE_CREATION_STATIC=y (2) is obviously correct. However, depending on (1) is not correct: if the user doesn't provide any custom permission table and custom device table, then ROOTFS_DEVICE_TABLES will be empty. So instead, move the addition of the package-defined static devices outside of condition (1), and have it only under condition (2). Reported-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
* fs/common.mk: allow user provided permissions to override packages permissionsGravatar Yann E. MORIN2018-12-041-2/+3
| | | | | | | | | Reported-by: Matthew Weber <matthew.weber@rockwellcollins.com> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Matthew Weber <matthew.weber@rockwellcollins.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
* fs/common.mk: rename FULL_DEVICE_TABLE to ROOTFS_FULL_DEVICES_TABLEGravatar Thomas Petazzoni2018-12-041-5/+5
| | | | | | | | | | | | | | | | In commit 6b50f988ad0a6044530b30344d52139da2994237 ("fs/common.mk: rename internal variable"), USERS_TABLE was renamed to ROOTFS_FULL_USERS_TABLE. This commit follows the same direction by renaming the FULL_DEVICE_TABLE variable to ROOTFS_FULL_DEVICE_TABLE. In addition, for consistency, the file itself is renamed full_device_table.txt. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
* fs/common.mk: rename internal variableGravatar Yann E. MORIN2018-12-031-4/+6
| | | | | | | | | | In preparation of more renames, rename the variable that points to the final users table. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> [Thomas: as suggested by Arnout, use ROOTFS_FULL_USERS_TABLE instead of ROOTFS_FINAL_USERS_TABLE.] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* Merge branch 'next'Gravatar Peter Korsgaard2018-12-021-5/+8
|\ | | | | | | Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
| * fs/common: allow filesystems to set the name of their output fileGravatar Carlos Santos2018-12-011-5/+7
| | | | | | | | | | | | | | | | | | | | | | Some filesystems may want to tweak their output names, rather than using the fixed "rootfs.foo" scheme. Add a ROOTFS_FOO_IMAGE_NAME variable for this purpose. Signed-off-by: Carlos Santos <casantos@datacom.com.br> [yann.morin.1998@free.fr: fix the patch] Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
| * Makefile: rework main directory creation logicGravatar Thomas Petazzoni2018-11-261-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In the current code, the creation of the main output directories (BUILD_DIR, STAGING_DIR, HOST_DIR, TARGET_DIR, etc.) is done by a global "dirs" target. While this works fine in the current situation, it doesn't work well in a context where per-package host and target directories are used. For example, with the current code and per-package host directories, the output/staging symbolic link ends up being created as a link to the per-package package sysroot directory of the first package being built, instead of the global sysroot. This commit reworks the creation of those directories by having the package/pkg-generic.mk code ensure that the build directory, target directory, host directory, staging directory and binaries directory exist before they are needed. Two new targets, host-finalize and staging-finalize are added in the main Makefile to create the compatibility symlinks for host and staging directories. They will be extended later with additional logic for per-package directories. Thanks to those changes, the global "dirs" target is entirely removed. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* | fs/tar: add support for xattrs (thus capabilties)Gravatar Yann E. MORIN2018-11-201-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | By default, tar will not include any extended attribute (xattr) when creating archives, and thus will not store capabilties either (as they are stored in the xattr 'security.capability'). Using option --xattrs is enough to create a tarball with all the xattrs attached to a file. However, extracting all xattrs from a tarball requires that --xattrs-include='*' be used. This is not symetric (but on purpose, as per the documentation), and so is confusing to some. So, we use --xattrs-include='*' to create the archive, so as to be explicit that we want all xattrs to be stored. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com> Cc: Peter Korsgaard <peter@korsgaard.com> Cc: Arnout Vandecappelle <arnout@mind.be> Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* | fs/common.mk: Fix show-build-orderGravatar Serj Kalichev2018-11-181-1/+1
| | | | | | | | | | | | | | | | | | | | The command "make show-build-order" doesn't show dependencies of rootfs-common target. This patch adds $(ROOTFS_COMMON_DEPENDENCIES) to PACKAGES variable. Signed-off-by: Serj Kalichev <serj.kalichev@gmail.com> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* | fs: don't use an intermediate tarballGravatar Yann E. MORIN2018-11-131-41/+18
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since 118534fe54b (fs: use a common tarball as base for the other filesystems), the filesystem creation is split in two steps, using an intermediate tarball to carry the generic, common finalisations to the per-filesystem finalisation and image creation. However, this intermediate tarball causes an issue with capabilities: they are entirely missing in the generated filesystems. Capabilities are stored in the extended attribute security.capability, which tar by default will not store/restore, unless explicitly told to, e.g. with --xattrs-include='*', which we don't pass. Now, passing this option when creating and extracting the intermediate tarball, both done under fakeroot, will cause fakeroot to report an invalid filetype for files with capabilities. mksquashfs would report such unknown files as a warning, while mkfs.ext2 would fail (with a similar error message), e.g.: File [...]/usr/sbin/getcap has unrecognised filetype 0, ignoring This is due to a poor interaction between tar and fakeroot; running as root the exact same commands we run under fakeroot, works as expected. Unfortunately, short of fixing fakeroot (which would first require understanding the problem in there), we don't have much options. The intermediate tarball was made to avoid redoing the same actions over and over again for each filesystem to build. However, most of the time, only one or two such filesystems would be enabled [0], and those actions are usually pretty lightweight. So, using an intermediate tarball does not provide a big optimisation. The main reason to introduce the intermediate tarball, however, is that it allows to postpone per-filesystem finalisations to be applied only for the corresponding filesystem, not for all of them. So, we get rid of the intermediate tarball, and simply move all of the code to run under fakeroot to the per-filesystem fakeroot script. Instead of extracting the intermediate tarball, we just rsync the original target/ directory, and apply the filesystem finalisations on that copy. The only thing still done in the rootfs-common step is to generate the intermediate files (users file, devices file) that are used in the fakeroot script. Fixes: https://bugs.busybox.net/show_bug.cgi?id=11216 Note: an alternate solution would have been to keep the intermediate tarball to keep most of the common finalisations, and move only the permissions to each filesystem, but that was getting a bit more complex and changed the ordering of permissions and post-fakeroot scripts. Once we bite the bullet of having some common finalisation done in each filesystem, it's easier to just move all of them. [0] Most probsably, users would enable the real filesystem to put on their device, plus the 'tar' filesystem, to be able to easily inspect the content on their development machine. Reported-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Peter Korsgaard <peter@korsgaard.com> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
* fs/f2fs: simplify overprovision option handlingGravatar Grzegorz Blach2018-11-081-5/+1
| | | | | | | | Setting overprovision to 0 and omitting this option has exactly the same effect. Signed-off-by: Grzegorz Blach <grzegorz@blach.pl> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* fs/f2fs: add option to define list of hot file extensionsGravatar Grzegorz Blach2018-11-082-0/+9
| | | | | Signed-off-by: Grzegorz Blach <grzegorz@blach.pl> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* fs/f2fs: clarify help messagesGravatar Grzegorz Blach2018-11-081-5/+6
| | | | | | | | | | | For BR2_TARGET_ROOTFS_F2FS_COLD_FILES clarify that extension list must be a coma separated. For BR2_TARGET_ROOTFS_F2FS_OVERPROVISION clarify that the default overprovision ratio is autocalculated according to the partition size. Signed-off-by: Grzegorz Blach <grzegorz@blach.pl> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* fs/f2fs/Config.in: remove consecutive empty linesGravatar Thomas Petazzoni2018-11-031-1/+0
| | | | | | | | This fixes the following check-package warning: fs/f2fs/Config.in:51: consecutive empty lines Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* fs/f2fs: add option to define list of filesystem featuresGravatar Grzegorz Blach2018-11-032-1/+11
| | | | | | Signed-off-by: Grzegorz Blach <grzegorz@blach.pl> [Thomas: split from the initial patch from Grzegorz] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* fs/f2fs: add option to define discard policyGravatar Grzegorz Blach2018-11-032-0/+13
| | | | | | Signed-off-by: Grzegorz Blach <grzegorz@blach.pl> [Thomas: split from the initial patch from Grzegorz] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* fs/f2fs: add option to define overprovision ratioGravatar Grzegorz Blach2018-11-032-1/+17
| | | | | | | Signed-off-by: Grzegorz Blach <grzegorz@blach.pl> [Thomas: split from the initial patch from Grzegorz, reworded Config.in help text] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* fs/f2fs: add option to define list of cold file extensionsGravatar Grzegorz Blach2018-11-032-1/+10
| | | | | | Signed-off-by: Grzegorz Blach <grzegorz@blach.pl> [Thomas: split from the initial patch from Grzegorz] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* fs/f2fs: add support for creating a f2fs imageGravatar Grzegorz Blach2018-11-033-0/+52
| | | | | | | | | | | | This patch makes possible to create rootfs image using f2fs filesystem. Signed-off-by: Grzegorz Blach <grzegorz@blach.pl> [Thomas: - keep only the minimal functionality, as suggested by Yann E. Morin - use truncate -s instead of dd to create the initial empty image file, as suggested by Yann E. Morin] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* fs/common: always depend on build host-tar if neededGravatar Yann E. MORIN2018-11-011-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, the filesystems do not depend on building host-tar when it is needed, even though all of them have to extract the intermediate tarball. However, in degenerate (but legally valid) configurations with no user-selectable package selected, host-tar would not be built, so the rootfs images would use whatever improper tar the system has. Add the conditional dependency to host-tar to the rootfs-common intermediate image. Since this is the internal step that all real rootfs generators depend on, they now properly depend on host-tar when needed. In practice, when host-tar is needed, it will always be built before the rootfs images, because it is a dependency of all packages (except a very few, like the skeleton), of which host-fakeroot, which is a mandatory dependency of rootfs-comon anyway. But for consistency sake, let's explicitly add host-tar as a dependency to rootfs-common too. Note that rootfs-tar already had that dependency, and we leave it as-is because it is semantically correct, even if superfluous. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* fs/common: allow custom user table to override package-defined usersGravatar Matt Weber2018-10-201-1/+2
| | | | | | | | | | | | | | | | | | | | | Currently, when a custom user table and a package define the same user, the settings from the package takes precedence over the ones from the custom user table. However, it makes sense to allow the settings from the custom user table take precedence. For example, it would allow redirecting the user's home directory to an alternate location (e.g. away from tmp and into a partition that is persistent). The support/scripts/mkusers script will only retain settings from the latest definition it finds. Thus, by passing the custom user table after the package defined users, it is possible to override the package provided user definitions. Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
* fs/btrfs: remove destination file before generationGravatar Yann E. MORIN2018-08-251-0/+1
| | | | | | | | | | | | | | | | | | | | btrfs will happily use an existing destination file if it already exists, increasing its size if needed. Hoever, it will never decrease the size, even if the requested size is smaller than the existing file. So, remove any previously existing destination file before generating the new filesystem. Note: the original submission by Robert did that, but as this case was not obvious, the removal was dropped by a refactoring when the patch was initially applied. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Robert J. Heywood <robert.heywood@codethink.co.uk> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> [Thomas: use $@ instead of $(@), use $(RM) instead of rm.] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* fs/btrfs: add support for generating a btrfs imageGravatar Robert J. Heywood2018-08-253-0/+86
| | | | | | | | | | | | | | | | | | | | | | | | | This patch makes it possible to format the rootfs using btrfs. It introduces the option; BR2_TARGET_ROOTFS_BTRFS. When selected, the user is able to specify the filesystem size, label, options, and node and sector sizes. The new files are based on fs/ext2/{Config.in,ext2.mk} Signed-off-by: Robert J. Heywood <robert.heywood@codethink.co.uk> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr> [Thomas: - fix issues pointed by Yann (duplicated empty line, missing quotes around default values for string options) - use -f option so that we don't have to remove the image file before creating it again - use the --byte-count option to set the filesystem size, which avoids the need for doing a "truncate -s" - remove the possible explanation of a mkfs.btrfs error. Indeed, mkfs.btrfs automatically extends the size of the image as needed, so the size passed can never be "too small". - fix check-package warnings in Config.in file.] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* fs: ensure hard links in TARGET_DIR are correctly copied for filesystem inputGravatar Peter Korsgaard2018-05-271-1/+1
| | | | | | | | | | | | | | | | | | | Fixes #11046 Commit bb2a57a17a7 (fs: run packages' filesystem hooks in a copy of target/) changed the file system logic to run file system hooks from packages on a copy of TARGET_DIR, and finally use this copy as input for the file system generation. This copy was done with rsync, which by default does not preserve hard links, leading to an expansion of the file system images when hard links are present. Fix it by passing the -H option to rsync (preserve hard links). Signed-off-by: Peter Korsgaard <peter@korsgaard.com> Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>