path: root/fs/common.mk
Commit message (Collapse)AuthorAgeFilesLines
* core: add make-based full-dependency listGravatar Yann E. MORIN2019-03-171-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, when we need to build the full dependency graph, we call make to show the list of packages (make show-targets), and then call it again and again iteratively while it returns new packages. Since calling make will parse the whole set of our Makefiles, this takes quite a bit of time (~4s each here), and the total can get pretty long. However, make being make, already builds the whole dependency tree information, so we can just ask for it. Add a new top-level rule 'show-dependency-tree' that displays the whole set of dependencies for all packages. For each package, its name, type and version is displayed, then all the direct, first-level dependencies are dumped. We choose a format that is not unlike the dot-graph format, because it is both easy to read as a human, and easy to parse as a machine: foo: target 1.2.3 foo -> bar host-meh bar: target virtual bar -> buz buz: target 2.3.4 buz -> host-meh: host virtual host-meh -> host-bleark host-bleark: host 3.4.5 host-bleark -> rootfs-meh: host rootfs-meh -> host-bleark To be noted: rootfs are currently reported as if they were 'host' packages, to stay aligned with how graph-depends currently treats them. Ideally, graph-depends could be enhanced to recognise them separately, but that is another story. For just plain defconfig, which is about the smallest config we can have with an internal toolchain, we already have a seven-fold improvement (with the graph-depends rule modified to not run the pdf generation, to be able to just compare the tree generation): $ time make graph-depends real 0m27.344s $ time make show-dependency-tree real 0m3.848s >From defconfig, C++, wchar, locales, ssp, and allyespackageconfig, tweaked for even more packages (qt5 not qt4, luajit to avoid multi providers, etc...), the timings are (graph-depends still modified to not generate the pdf): $ time make graph-depends real 1m56.459s $ time make show-dependency-tree real 0m5.748s There. I don't think those numbers need any explanation whatsoever; they do speak on their own. OK, for maths sake, the ratio is about twenty-fold. So, "yeah", I guess... ;-) Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* fs/common.mk: make sure that static devices from packages are createdGravatar Thomas Petazzoni2018-12-041-1/+1
| | | | | | | | | | | | | | | | | | | The static devices defined by packages are currently added to the full device table when two conditions are met: (1) ROOTFS_DEVICE_TABLES is non-empty (2) BR2_ROOTFS_DEVICE_CREATION_STATIC=y (2) is obviously correct. However, depending on (1) is not correct: if the user doesn't provide any custom permission table and custom device table, then ROOTFS_DEVICE_TABLES will be empty. So instead, move the addition of the package-defined static devices outside of condition (1), and have it only under condition (2). Reported-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
* fs/common.mk: allow user provided permissions to override packages permissionsGravatar Yann E. MORIN2018-12-041-2/+3
| | | | | | | | | Reported-by: Matthew Weber <matthew.weber@rockwellcollins.com> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Matthew Weber <matthew.weber@rockwellcollins.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
* fs/common.mk: rename FULL_DEVICE_TABLE to ROOTFS_FULL_DEVICES_TABLEGravatar Thomas Petazzoni2018-12-041-5/+5
| | | | | | | | | | | | | | | | In commit 6b50f988ad0a6044530b30344d52139da2994237 ("fs/common.mk: rename internal variable"), USERS_TABLE was renamed to ROOTFS_FULL_USERS_TABLE. This commit follows the same direction by renaming the FULL_DEVICE_TABLE variable to ROOTFS_FULL_DEVICE_TABLE. In addition, for consistency, the file itself is renamed full_device_table.txt. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
* fs/common.mk: rename internal variableGravatar Yann E. MORIN2018-12-031-4/+6
| | | | | | | | | | In preparation of more renames, rename the variable that points to the final users table. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> [Thomas: as suggested by Arnout, use ROOTFS_FULL_USERS_TABLE instead of ROOTFS_FINAL_USERS_TABLE.] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* Merge branch 'next'Gravatar Peter Korsgaard2018-12-021-5/+8
|\ | | | | | | Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
| * fs/common: allow filesystems to set the name of their output fileGravatar Carlos Santos2018-12-011-5/+7
| | | | | | | | | | | | | | | | | | | | | | Some filesystems may want to tweak their output names, rather than using the fixed "rootfs.foo" scheme. Add a ROOTFS_FOO_IMAGE_NAME variable for this purpose. Signed-off-by: Carlos Santos <casantos@datacom.com.br> [yann.morin.1998@free.fr: fix the patch] Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
| * Makefile: rework main directory creation logicGravatar Thomas Petazzoni2018-11-261-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In the current code, the creation of the main output directories (BUILD_DIR, STAGING_DIR, HOST_DIR, TARGET_DIR, etc.) is done by a global "dirs" target. While this works fine in the current situation, it doesn't work well in a context where per-package host and target directories are used. For example, with the current code and per-package host directories, the output/staging symbolic link ends up being created as a link to the per-package package sysroot directory of the first package being built, instead of the global sysroot. This commit reworks the creation of those directories by having the package/pkg-generic.mk code ensure that the build directory, target directory, host directory, staging directory and binaries directory exist before they are needed. Two new targets, host-finalize and staging-finalize are added in the main Makefile to create the compatibility symlinks for host and staging directories. They will be extended later with additional logic for per-package directories. Thanks to those changes, the global "dirs" target is entirely removed. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* | fs/common.mk: Fix show-build-orderGravatar Serj Kalichev2018-11-181-1/+1
| | | | | | | | | | | | | | | | | | | | The command "make show-build-order" doesn't show dependencies of rootfs-common target. This patch adds $(ROOTFS_COMMON_DEPENDENCIES) to PACKAGES variable. Signed-off-by: Serj Kalichev <serj.kalichev@gmail.com> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* | fs: don't use an intermediate tarballGravatar Yann E. MORIN2018-11-131-41/+18
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since 118534fe54b (fs: use a common tarball as base for the other filesystems), the filesystem creation is split in two steps, using an intermediate tarball to carry the generic, common finalisations to the per-filesystem finalisation and image creation. However, this intermediate tarball causes an issue with capabilities: they are entirely missing in the generated filesystems. Capabilities are stored in the extended attribute security.capability, which tar by default will not store/restore, unless explicitly told to, e.g. with --xattrs-include='*', which we don't pass. Now, passing this option when creating and extracting the intermediate tarball, both done under fakeroot, will cause fakeroot to report an invalid filetype for files with capabilities. mksquashfs would report such unknown files as a warning, while mkfs.ext2 would fail (with a similar error message), e.g.: File [...]/usr/sbin/getcap has unrecognised filetype 0, ignoring This is due to a poor interaction between tar and fakeroot; running as root the exact same commands we run under fakeroot, works as expected. Unfortunately, short of fixing fakeroot (which would first require understanding the problem in there), we don't have much options. The intermediate tarball was made to avoid redoing the same actions over and over again for each filesystem to build. However, most of the time, only one or two such filesystems would be enabled [0], and those actions are usually pretty lightweight. So, using an intermediate tarball does not provide a big optimisation. The main reason to introduce the intermediate tarball, however, is that it allows to postpone per-filesystem finalisations to be applied only for the corresponding filesystem, not for all of them. So, we get rid of the intermediate tarball, and simply move all of the code to run under fakeroot to the per-filesystem fakeroot script. Instead of extracting the intermediate tarball, we just rsync the original target/ directory, and apply the filesystem finalisations on that copy. The only thing still done in the rootfs-common step is to generate the intermediate files (users file, devices file) that are used in the fakeroot script. Fixes: https://bugs.busybox.net/show_bug.cgi?id=11216 Note: an alternate solution would have been to keep the intermediate tarball to keep most of the common finalisations, and move only the permissions to each filesystem, but that was getting a bit more complex and changed the ordering of permissions and post-fakeroot scripts. Once we bite the bullet of having some common finalisation done in each filesystem, it's easier to just move all of them. [0] Most probsably, users would enable the real filesystem to put on their device, plus the 'tar' filesystem, to be able to easily inspect the content on their development machine. Reported-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Peter Korsgaard <peter@korsgaard.com> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
* fs/common: always depend on build host-tar if neededGravatar Yann E. MORIN2018-11-011-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, the filesystems do not depend on building host-tar when it is needed, even though all of them have to extract the intermediate tarball. However, in degenerate (but legally valid) configurations with no user-selectable package selected, host-tar would not be built, so the rootfs images would use whatever improper tar the system has. Add the conditional dependency to host-tar to the rootfs-common intermediate image. Since this is the internal step that all real rootfs generators depend on, they now properly depend on host-tar when needed. In practice, when host-tar is needed, it will always be built before the rootfs images, because it is a dependency of all packages (except a very few, like the skeleton), of which host-fakeroot, which is a mandatory dependency of rootfs-comon anyway. But for consistency sake, let's explicitly add host-tar as a dependency to rootfs-common too. Note that rootfs-tar already had that dependency, and we leave it as-is because it is semantically correct, even if superfluous. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* fs/common: allow custom user table to override package-defined usersGravatar Matt Weber2018-10-201-1/+2
| | | | | | | | | | | | | | | | | | | | | Currently, when a custom user table and a package define the same user, the settings from the package takes precedence over the ones from the custom user table. However, it makes sense to allow the settings from the custom user table take precedence. For example, it would allow redirecting the user's home directory to an alternate location (e.g. away from tmp and into a partition that is persistent). The support/scripts/mkusers script will only retain settings from the latest definition it finds. Thus, by passing the custom user table after the package defined users, it is possible to override the package provided user definitions. Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
* fs: ensure hard links in TARGET_DIR are correctly copied for filesystem inputGravatar Peter Korsgaard2018-05-271-1/+1
| | | | | | | | | | | | | | | | | | | Fixes #11046 Commit bb2a57a17a7 (fs: run packages' filesystem hooks in a copy of target/) changed the file system logic to run file system hooks from packages on a copy of TARGET_DIR, and finally use this copy as input for the file system generation. This copy was done with rsync, which by default does not preserve hard links, leading to an expansion of the file system images when hard links are present. Fix it by passing the -H option to rsync (preserve hard links). Signed-off-by: Peter Korsgaard <peter@korsgaard.com> Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* fs: remove intermediate artefactsGravatar Yann E. MORIN2018-03-311-0/+2
| | | | | | | | | | | | | | | | | | | | Each of the intermediate, per-rootfs target directories, as well as the intermediate tarball, can take quite some place, and is mostly a duplication of what's already in target/. The only delta, if any, would be the tweaks made by the filesystem image generations, but those tweaks are most probably only meaningful when seen as root. We normally do not remove intermediate files, but those can be quite large, and are not directly usable by, nor accessible to the user. So, get rid of them once the filesystem has been generated. This does not need to be done in fakeroot. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Arnout Vandecappelle <arnout@mind.be> Tested-by: Matt Weber <matthew.weber@rockwellcollins.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* fs: get rid of package-provided post-fs hooksGravatar Yann E. MORIN2018-03-311-2/+0
| | | | | | | | | | | | | | | | | Now that the pre-fs ones are run on a transient copy of target/, the post-fs hooks are no longer needed because we no longer need to restore the target/ directory as it is only a internal copy. Remove support for the post-fs hooks, and update the sole package using them. We do not add a legacy check because this was mostly a purely-internal detail that was never really exposed nor documented. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Arnout Vandecappelle <arnout@mind.be> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* fs: run packages' filesystem hooks in a copy of target/Gravatar Yann E. MORIN2018-03-311-2/+5
| | | | | | | | | | | | | | | | | | | | Currently, some packages may register hooks to be run just before and just after the generic tarball image is generated, because they need to prepare the filesystem for read-only or read-write operation. However, this means that, if any of the hooks or the image generation fails, the target directory is left in a dangling, inconsistent state. We fix that by doing a copy of target/, run the hooks on that copy, generate the generic tarball image out of that, and get rid of the copy. This way, we can guarantee consistency of the target directory, and we can even ditch support for post-fs hooks (those that restore target/). Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Arnout Vandecappelle <arnout@mind.be> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* fs: use a common tarball as base for the other filesystemsGravatar Yann E. MORIN2018-03-311-28/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, some filesystems may want to tweak the content of the target directory, create special device nodes etc... This all means that: - the content of the target directory for a specific filesystems may depend on whether another filesystem is enabled or not; for example, cpio will create a /init script or symlink and a /dev/console node; - the filesystems can not be built in parallel, because they may change the content of the target directory while another is being assembled. Furthermore, the same fakeroot script is executed over-and-over-again for each filesystem, to create the device nodes, the users and their homes and files, and setting permissions... We introduce an intermediate tarball, for which we do the full fakeroot shebang. That tarball then serves as the base for the other filesystems, with a very simple fakeroot script that untars the common tarball, and calls the actual filesystem image generator on that. Note that we use a very simple tar command to generate the intermediate tarball, because we are not concerned with reproducibility of the archive itself (only of the archived files). Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Arnout Vandecappelle <arnout@mind.be> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* fs: introduce a macro for reproducible commandGravatar Yann E. MORIN2018-03-311-3/+7
| | | | | | | Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Arnout Vandecappelle <arnout@mind.be> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* fs: new intermediate rootfs-common to gather common dependenciesGravatar Yann E. MORIN2018-03-311-4/+19
| | | | | | | | | | | | | | | Before we can create an intermediate tarball for all filesystems, we nedd to move the common dependencies needed to generate that intermediate tarball, rather than leave those dependencies to each filesystem. So, we introduce rootfs-common, which gathers all those common dependencies. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Arnout Vandecappelle <arnout@mind.be> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* fs: introduce per-rootfs TARGET_DIR variableGravatar Yann E. MORIN2018-03-311-0/+1
| | | | | | | | | | ... which for now still points to the base target directory, but this is a step forward. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Arnout Vandecappelle <arnout@mind.be> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* fs: split per-rootfs dependency lineGravatar Yann E. MORIN2018-03-311-1/+2
| | | | | | | Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Arnout Vandecappelle <arnout@mind.be> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* fs: use a per-rootfs fakeroot scriptGravatar Yann E. MORIN2018-03-311-3/+3
| | | | | | | | | | | | | | ... and locate that script in a per-rootfs directory. Just like for ROOTFS, this variable will leak down the dependency tree to target-finalize and packages - But it doesn't matter as it isn't used outside fs/. [Peter: extend commit message] Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Arnout Vandecappelle <arnout@mind.be> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* fs: define per-rootfs directoryGravatar Yann E. MORIN2018-03-311-0/+2
| | | | | | | | | | This will serve in future commits to store pre-rootfs files, like fakeroot script... Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Arnout Vandecappelle <arnout@mind.be> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* fs: set per-rootfs variable nameGravatar Yann E. MORIN2018-03-311-0/+1
| | | | | | | | | | | | | | | | | | | | Like we do for packages with the PKG variable, set ROOTFS to contain the upper-case name of the rootfs currently being generated. This will be useful in later patches, when we need more per-rootfs variables, like a per-rootfs TARGET_DIR for example. In Makefiles, per-rule variables trickle down the dependency chain, to all dependencies of that rule, so we have to stop ROOTFS as soon as we're not in a rootfs. This means we have to stop it at target-finalize (which is a dependency of all filesystems), and for each package individually, since some packages (host or target) can be direct dependencies of filesystems as well. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Arnout Vandecappelle <arnout@mind.be> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* fs: run filesystem hooks under fakerootGravatar Yann E. MORIN2018-03-311-1/+2
| | | | | | | | | | | | | | | | | | Currently, the pre-gen hooks of the various filesystems are run before we enter the fakeroot. However, this precludes those hooks from doing actions that require root, like creating a pseudo-device or the likes. So, move those pre-gen hooks under fakeroot. This has currently no side-effect, as they are still called before everything else in the fakeroot script, even the system-wide chown call. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Arnout Vandecappelle <arnout@mind.be> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* fs: pass EXTRA_ENV to post-fakeroot scriptGravatar Yann E. MORIN2018-01-151-1/+1
| | | | | | | | | Like we do for post-build and post-image scripts, pass EXTRA_ENV to post-fakeroot script. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* fs/common.mk: support lz4 compressionGravatar Peter Korsgaard2018-01-081-0/+5
| | | | | | | | | | Similar to the other compressors. Notice that we use the -l (legacy format) for Linux kernel initrd compatibility. Lz4 decompression is supported by the Linux kernel since 3.11. Signed-off-by: Peter Korsgaard <peter@korsgaard.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* fs: make it behave a bit more like the package infraGravatar Yann E. MORIN2018-01-031-4/+3
| | | | | | | | | | | | | | Currently, to register a filesystem, one has to call: $(eval $(call ROOTFS_TARGET,blabla)) This is very unlike the package infrastructure, where the name of the package is automatically guessed by the infra. It turns out that we can now do that for the filesystem infra too. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* fs: prepare temp directory before running PRE_GEN hooksGravatar Yann E. MORIN2017-12-021-1/+1
| | | | | | | | | | | | | | | | | | | Some filesystems have PRE_GEN hooks that create a directory structure under the temporary directory. For example, iso9660 will create a sub-directory where it stores the kernel (in case of initramfs or initrd). So, we must run the PRE_GEN hooks after we cleanup/create the temporary directory. Fixes: https://gitlab.com/buildroot.org/buildroot/-/jobs/42835965 https://gitlab.com/buildroot.org/buildroot/-/jobs/42835967 Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* fs: don't remove intermediate filesGravatar Yann E. MORIN2017-11-301-1/+0
| | | | | | | | | | | | Now that they are in their own directory and no longer pollute the build dir, there is no point in removing them. Furthermore, a follow-up patch will require that those files survive when more than one filesystem image is generated. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* fs: don't pollute $(BUILD_DIR) with temp filesGravatar Yann E. MORIN2017-11-301-5/+6
| | | | | | | | | | | Currently, we create a bunch of temporary files in $(BUILD_DIR), while assembling the filesystem images. Move those files to their own sub-directory. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* fs: remove TARGET_DIR_WARNING_FILE lateGravatar Yann E. MORIN2017-11-301-1/+1
| | | | | | | | | | Remove it just before generating the filesystem image. This way, removing-and-recreating the file encloses the actual image generation as tightly as possible. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* fs/common: get rid of post-target rulesGravatar Yann E. MORIN2017-11-301-5/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | The only users of post-target rules were ext2, cpio and initramfs. Of those, ext2 and cpio were changed to use post-gen hooks, while initramfs was not even using the generic rootfs infra and was fixed to no longer reference post-target rules. Besides, the comment in the infra was really misleading: it referenced initramfs implying it was the sole user of that feature, even though initramfs was not using the fs infra. Furthermore, using post-target rules was inherently broken for top-level parallel builds, because filesystems had to ensure the ordering by themselves. Of the two real users of post-target rules (cpio and ext2), one did enforce rules ordering (apparently correctly), while the other forgot to do so. We can get rid of post-target rules altogether, now. Add a legacy check, to catch out-of-tree (e.g. br2-external) users of post-target rules, and instruct them to switch to post-gen hooks instead. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Arnout Vandecappelle <arnout@mind.be> Reviewed-by: Romain Naour <romain.naour@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* fs: add missing $$(sep) to pre- and post-command hooks codeGravatar Andrey Smirnov2017-11-051-2/+2
| | | | | | | | | | | | | | | When multiple hooks are registred, both pre-a and post-command hooks' foreach loops need to have a separator at the end in order for the code to work as intended. Without the separator all hooks end up as a one single line command thus making all but the first hook into no-ops. Fixes: 4628b6f3b4 ("fs: add pre- and post-command hooks") Cc: Yann E. MORIN <yann.morin.1998@free.fr> Cc: Romain Naour <romain.naour@gmail.com> Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com> Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
* fs: add pre- and post-command hooksGravatar Yann E. MORIN2017-08-021-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In some cases, the directory structure we want in the filesystem is not exactly what we have in target/ For example, when systemd is used on a read-only rootfs, /var must be a tmpfs. However, we may have packages that install stuff in there, and set important rights (via the permission-table). So, at build time, we need /var to be a symlink to the remanent location (/usr/share/factory) while at runtime we need /var to be a directory. One option would have been to have /var as a real directory even during build time, and in a target-finalize hook, move everything out of there and into the "factory" location. However, that's not possible because it's too early: some packages may want to set ownership and/or acces rights on directories or files in /var, and this is only done in the fakeroot script, which is called only later during the assembling of the filesystem images. Also, there would have been no way to undo the tweak (i.e. we need to restore the /var symlink so that subsequent builds continue to work) if it were done as a target-finalize hook. The only solution is to allow packages to register pre- and post-hooks that are called right before and right after the rootfs commands are executed, and inside in the fakeroot script. We can however not re-use the BR2_ROOTFS_POST_FAKEROOT_SCRIPT feature either because it is done before the filesystem command, but there is nothing that is done after. Also, we don't want to add to, and modify a user-supplied variable. So, we introduce two new variables that packages can set to add the commands they need to run to tweak the filesystem right at the last moment. Those hooks are not documented on-purpose; they are probably going to only ever be used by systemd. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Reviewed-by: Romain Naour <romain.naour@gmail.com> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* Globally replace $(HOST_DIR)/usr/bin with $(HOST_DIR)/binGravatar Arnout Vandecappelle2017-07-051-2/+2
| | | | | | | | | | | Since things are no longer installed in $(HOST_DIR)/usr, the callers should also not refer to it. This is a mechanical change with git grep -l '$(HOST_DIR)/usr/bin' | xargs sed -i 's%$(HOST_DIR)/usr/bin%$(HOST_DIR)/bin%g' Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* fs: set packages permissions even with no system device tablesGravatar Yann E. MORIN2016-12-141-1/+1
| | | | | | | | | | | | | | | | | | Currently, when there is no syztem device table (permissions or static devices) defined, then package permissions are not applied, because they are guarded by the check on the system device tables being non empty. Fix that by narrowing the guarding condition. Note that the dependency on host-makedevs was not conditional; we always build it even if we don't need it. Making it conditional is not possible, because we don't know all the packages permissions by the time the fs infra is parsed (packages from br2-external are parsed after it). Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Peter Korsgaard <peter@korsgaard.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* Merge branch 'next'Gravatar Peter Korsgaard2016-12-011-0/+3
|\ | | | | | | Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
| * fs/common: lock modification times in $TARGET_DIRGravatar Jérôme Pouiller2016-11-231-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | Make sure all files in $TARGET_DIR have a defined modification time before to generate filesystems. This work was sponsored by `BA Robotic Systems'. Signed-off-by: Jérôme Pouiller <jezz@sysmic.org> Reviewed-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* | Revert "fs: use pseudo instead of fakeroot"Gravatar Peter Korsgaard2016-11-271-2/+2
| | | | | | | | | | | | | | | | | | | | This reverts commit 8035ceb56cc3ba2541aba0d0ead3f29900d9e310. Moving to pseudo brought a number of issues (and longer compilation time), so lets stick with fakeroot now that the reported ubi issue has been worked around. Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* | Revert "fs: call the pseudo wrapper"Gravatar Peter Korsgaard2016-11-271-1/+1
| | | | | | | | | | | | This reverts commit 74a3e75bb8208a8f3bb8426549fce1ea09b3be8a. Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* | fs: call the pseudo wrapperGravatar Yann E. MORIN2016-11-091-1/+1
|/ | | | | | | | | | | Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Gaël PORTAY <gael.portay@savoirfairelinux.com> Cc: Patrick Keroulas <patrick.keroulas@savoirfairelinux.com> Cc: Erico Nunes <nunes.erico@gmail.com> Cc: Julien BOIBESSOT <julien.boibessot@free.fr> Cc: Arnout Vandecappelle <arnout@mind.be> Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* fs: use pseudo instead of fakerootGravatar Yann E. MORIN2016-11-031-2/+2
| | | | | | | | | | | | | | | | | fakeroot misbhaves when there are SElinux contexts, as reported in #9386, and further detailed in https://bugzilla.redhat.com/show_bug.cgi?id=1238802 A proposal in the FC bug is to use pseudo instead of fakeroot. Pseudo is a from-scratch re-implementation of fakeroot; it is used in Poky/OE instead of fakeroot. Fixes #9386. Reported-by: Andrey Yurovsky <yurovsky@gmail.com> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Andrey Yurovsky <yurovsky@gmail.com> Cc: Gustavo Zacarias <gustavo@zacarias.com.ar> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* fs/common: add option to execute custom scripts under fakerootGravatar Yann E. MORIN2016-07-031-0/+3
| | | | | | | | | | | | | | | | Some users have the need to be able to tweak the content of the target rootfs with root-like rights, that is, from inside the fakeroot script. Add a new system option to allow those users to provide a list of scripts, like the post-build and post-image scripts, that will be run from our fakeroot script. [Peter: pass TARGET_DIR to scripts, tweak help text] Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Peter Korsgaard <peter@korsgaard.com> Cc: Cam Hutchison <camh@xdna.net> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* fs: add sha-bang to fakeroot scriptGravatar Yann E. MORIN2016-06-071-0/+1
| | | | | | | | We directly running this script, so it should start with a sha-bang (not sure why/how it works today...). Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* fs: properly escape commands when generating fakeroot scriptGravatar Yann E. MORIN2016-06-071-4/+4
| | | | | | | | Use the newly-introduced PRINTF macro to generate printf formats that do "The Right Thing (TM)". Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* fs/common: fix typo in commentsGravatar Brian 'redbeard' Harrington2016-05-081-1/+1
| | | | | | | Fixed comment typos as to improve clarity. Signed-off-by: Brian 'redbeard' Harrington <redbeard@coreos.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* fs/common: generate users before setting permissionsGravatar Yann E. MORIN2016-02-011-5/+5
| | | | | | | | | We will need the users and groups to get defined before we can use them from makedevs. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* fs/common: add <fs>_POST_GEN_HOOKS supportGravatar Thomas Petazzoni2015-08-101-0/+4
| | | | | | | | | | | | | | | | | The common filesystem infrastructure already supports a <fs>_PRE_GEN_HOOKS variable, which allows filesystem makefiles to register some actions to be done before the root filesystem image is generated. This commit adds a similiar <fs>_POST_GEN_HOOKS variable, which will allow filesystem makefiles to do some actions after the filesystem image has been generated. It will initially be used by the iso9660 filesystem to delete the temporary directory it creates. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Signed-off-by: Noé Rubinstein <nrubinstein@aldebaran.com> Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* fs: build host-mkpasswd when BR2_ROOTFS_USERS_TABLES is usedGravatar Thomas Petazzoni2015-07-211-1/+1
| | | | | | | | | | | | | | | | | | | The code in fs/common.mk properly triggers the build of host-mkpasswd when at least one package specifies a <pkg>_USERS variable. However, when no selected package specifies a <pkg>_USERS variable but the user uses a custom users table through BR2_ROOTFS_USERS_TABLES, then we forget to build host-mkpasswd, leading to build failures if you don't have mkpasswd installed in your machine. This commit fixes that by taking into account both the package users table and the custom users table when deciding to depend or not on host-mkpasswd. Reported-by: Auke Willem Oosterhoff <oosterhoff@baopt.nl> Cc: Auke Willem Oosterhoff <oosterhoff@baopt.nl> Cc: Yann E. MORIN <yann.morin.1998@free.fr> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>