aboutsummaryrefslogtreecommitdiff
path: root/support
Commit message (Collapse)AuthorAgeFilesLines
* support/scripts/get-developers: add new scriptGravatar Thomas Petazzoni2016-09-212-0/+284
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This script, and its companion library, is more-or-less Buildroot's equivalent to the kernel get_maintainer.pl script: it allows to get the list of developers to whom a set of patches should be sent to. To do so, it first relies on a text file, named DEVELOPERS, at the root of the Buildroot source tree (added in a followup commit) to list the developers and the files they are interested in. The DEVELOPERS file's format is simple: N: Firstname Lastname <email> F: path/to/file F: path/to/another/file This allows to associate developers with the files they are looking after, be they related to a package, a defconfig, a filesystem image, a package infrastructure, the documentation, or anything else. When a directory is given, the tool assumes that the developer handles all files and subdirectories in this directory. For example "package/qt5/" can be used for the developers looking after all the Qt5 packages. Conventional shell patterns can be used, so "package/python-*" can be used for the developers who want to look after all packages matching "python-*". A few files are recognized specially: - .mk files are parsed, and if they contain $(eval $(<something>-package)), the developer is assumed to be looking after the corresponding package. This way, autobuilder failures for this package can be reported directly to this developer. - arch/Config.in.<arch> files are recognized as "the developer is looking after the <arch> architecture". In this case, get-developer parses the arch/Config.in.<arch> to get the list of possible BR2_ARCH values. This way, autobuilder failures for this package can be reported directly to this developer. - pkg/pkg-<infra>.mk are recognized as "the developer is looking after the <infra> package infrastructure. In this case, any patch that adds or touches a .mk file that uses this infrastructure will be sent to this developer. Examples of usage: $ ./support/scripts/get-developers 0001-ffmpeg-fix-bfin-build.patch git send-email--to buildroot@buildroot.org --to "Luca Ceresoli <luca@lucaceresoli.net>" --to "Bernd Kuhls <bernd.kuhls@t-online.de>" $ ./support/scripts/get-developers -p imx-lib Arnout Vandecappelle <arnout@mind.be> Gary Bisson <gary.bisson@boundarydevices.com> $ ./support/scripts/get-developers -a bfin Waldemar Brodkorb <wbx@openadk.org> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* manual: get rid of the lists of packagesGravatar Yann E. MORIN2016-09-193-4084/+0
| | | | | | | | | | | | | | | | | | | | | | | | | We currently have four lists of packages in the manual: - the non-virtual target packages, - the virtual target packages, - the host packages, - the deprecated features. Those list take more than half of the manual. They do not serve much purpose except to show off. After the recent discussion on the list [0], remove them all. We can now get rid of our biggish and complex generating script (and its companion library kconfiglib). [0] http://lists.busybox.net/pipermail/buildroot/2016-September/171199.html Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Peter Korsgaard <peter@korsgaard.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Samuel Martin <s.martin49@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* core: don't build host-cmake if it is available on the build hostGravatar Luca Ceresoli2016-09-172-0/+57
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently all cmake packages depend on host-cmake. Unfortunately host-cmake takes a long time to configure and build: almost 7 minutes on a dual-core i5 with SSD. The time does not change even with ccache enabled. Indeed, building host-cmake is avoidable if it is already installed on the build host: CMake is supposed to be quite portable, and the only patch in Buildroot for the CMake package seems to only affect target-cmake. Thus we automatically skip building host-cmake and use the one on the system if: - cmake is available on the system and - it is recent enough. First, we leverage the existing infrastructure in support/dependencies/dependencies.mk to find out whether there's a suitable cmake executable on the system. Its path can be passed in the BR2_CMAKE environment variable, otherwise it defaults to "cmake". If it is enabled, found and suitable then we set BR2_CMAKE_HOST_DEPENDENCY to empty; otherwise we set BR2_CMAKE_HOST_DEPENDENCY to 'host-cmake' and override BR2_CMAKE with "$(HOST_DIR)/usr/bin/cmake" to revert to using our own cmake (the old behaviour). Then in pkg-cmake.mk we replace the hard-coded dependency on host-cmake to using the BR2_CMAKE_HOST_DEPENDENCY variable, and we use $(BR2_CMAKE) instead of $(HOST_DIR)/usr/bin/cmake. Unlike what we do for host-tar and host-xzcat, for host-cmake we do not add host-cmake to DEPENDENCIES_HOST_PREREQ. If we did, host-cmake would be a dependency for _any_ package when it's not installed on the host, even when no cmake package is selected. Cmake versions older than 3.0 are affected by the bug described and fixed in Buildroot in ef2c1970e4bf ("cmake: add patch to fix Qt mkspecs detection"). The bug was fixed in upstream CMake in version 3.0 [0]. Amongst all the cmake packages currently in Buildroot, the currently highest version mentioned in cmake_minimum_required() is 3.1 (grantlee and opencv3). Thus we use 3.1 as the lowest required cmake for now, until a package is bumped, or a new package added, with a higher required version. [0] https://cmake.org/gitweb?p=cmake.git;h=e8b8b37ef6fef094940d3384df5a1d421b9fa568 Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net> Cc: Samuel Martin <s.martin49@gmail.com> Cc: Davide Viti <zinosat@tiscali.it> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Reviewed-by: Romain Naour <romain.naour@gmail.com> Tested-by: Romain Naour <romain.naour@gmail.com> [yann.morin.1998@free.fr: - simplify logic in check-host-cmake.mk; - set and use BR2_CMAKE_HOST_DEPENDENCY, drop USE_SYSTEM_CMAKE; - bump to cmake 3.1 for grantlee and opencv; ] Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* Merge branch 'next'Gravatar Peter Korsgaard2016-09-029-9/+136
|\ | | | | | | | | | | Quite some conflicts, so here goes .. Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
| * core: move pkg-utils.mk to support/Gravatar Yann E. MORIN2016-08-271-0/+111
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | pkg-utils.mk contains various definitions that are used in the package infrastructures and packages themselves. However, those definitions can be useful in other parts of Buildroot, and are already used in a few places that are not related to the package infrastructure. Also, $(sep) will be needed early in the Makefile when we eventually support multiple br2-external trees. Since this file only contains definitions, we can include it anytime. So, consider that file to no longer be specific to the package infras: - move it to support and rename it, - move a few similar definitions from the main Makefile to that file. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Peter Korsgaard <jacmet@uclibc.org> Reviewed-by: Romain Naour <romain.naour@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
| * support/download: Add support to pass options directly to downloadersGravatar Romain Perier2016-08-238-9/+25
| | | | | | | | | | | | | | | | | | | | | | This adds support to pass options to the underlying command that is used by downloader. Useful for retrieving data with server-side checking for user login or passwords, use a proxy or use specific options for cloning a repository via git and hg. Signed-off-by: Romain Perier <romain.perier@free-electrons.com> Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* | Update for 2016.082016.08Gravatar Peter Korsgaard2016-09-011-1/+1
| | | | | | | | Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* | docs: libtool patches are not handled by legal-info infrastructure.Gravatar Romain Naour2016-08-191-0/+5
| | | | | | | | | | | | | | | | | | | | Signed-off-by: Romain Naour <romain.naour@gmail.com> Cc: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Arnout Vandecappelle <arnout@mind.be> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> [Thomas: slightly reword with Yann's suggestion.] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* | pkg-cmake.mk: set CMAKE_SYSROOT variable in toolchainfile.cmakeGravatar Samuel Martin2016-08-181-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change enforces the CMAKE_SYSROOT value set in the toolchainfile.cmake. This fix overrides the CMake heuristics used to guess it, and turns off some non-desirable behavior adding "-isystem ..." flags to the compiler command line, misleading the compiler and making the build failed due to some unfound standard headers. Fixes: http://autobuild.buildroot.net/results/f7e/f7e92678e91a6cb15ccf32d4a7d75b39f49d6000/defconfig (and others) Cc: Ben Boeckel <mathstuf@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Signed-off-by: Samuel Martin <s.martin49@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* | size-stats: fix bug when parsing file names with commaGravatar Thomas Petazzoni2016-08-091-1/+1
|/ | | | | | | | | | | | | | | | | | | The size-stats script fails when the usb_modeswitch_data is enabled, because this package installs files that contain commas in their name. However, the size-stats script also uses comma as a separator for its CSV files, causing a "ValueError: too many values to unpack" in: pkg, fpath = l.split(",") Fix this by splitting only the two fields that need to be split. The bug was reported by Matthias <porto.rio@gmx.net>, who also suggested a fix. Fixes bug #9136. Reported-by: Matthias <porto.rio@gmx.net> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/download/git: Fix compatibility issue with git older than 1.8.4Gravatar Enrique Ocaña González2016-07-281-1/+1
| | | | | | | | | | | | The "--no-patch" option used by the git downloader appeared on git 1.8.4. Systems with older git versions show an error and fall back to the wget downloader, which isn't suitable for all the cases. Signed-off-by: Enrique Ocaña González <eocanha@igalia.com> Tested-by: Matthew Weber <matthew.weber@rockwellcollins.com> Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* support: pkg-stats: add efl package in packages statsGravatar Romain Naour2016-07-241-1/+0
| | | | | | | | Since efl update to 1.15 version, the efl package is a "real" Buildroot package. It doesn't contain any subdirectories anymore. Signed-off-by: Romain Naour <romain.naour@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* scanpypi: include LICENCE in the list of supported license filesGravatar Yegor Yefremov2016-07-071-1/+2
| | | | | Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/apply-patches: re-instate set -eGravatar Yann E. MORIN2016-07-061-1/+3
| | | | | | | | | | | | | | | | | | | As reported by Sébastien Szymanski [1], the apply-patches script doesn't stop if a tar command can't extract an archive. Use "set -e" to exit immediately if a command return an error. Be sure to ignore any expected error: when we check if a patch to be applied has the same basename as an already applied patch, the grep would fail when no such patch was already applied. We should not fail in this case. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Sébastien Szymanski <sebastien.szymanski@armadeus.com> Cc: Romain Naour <romain.naour@openwide.fr> Reviewed-by: Romain Naour <romain.naour@gmail.com> Tested-by: Romain Naour <romain.naour@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* Revert "apply-patches: catch unexpected failure"Gravatar Thomas Petazzoni2016-07-051-3/+0
| | | | | | | | This reverts commit 9cf1ad6cdb37c5be5772121896f4b13ec42c2b0d. This breaks many packages. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* apply-patches: catch unexpected failureGravatar Romain Naour2016-07-051-0/+3
| | | | | | | | | | | | | As reported by Sébastien Szymanski [1], the apply-patches script doesn't stop if a tar command can't extract an archive. Use "set -e" to exit immediately if a command return an error. [1] http://patchwork.ozlabs.org/patch/626196 Signed-off-by: Romain Naour <romain.naour@gmail.com> Cc: Sébastien Szymanski <sebastien.szymanski@armadeus.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* scancpan: improve message when bad host perl versionGravatar Francois Perrad2016-07-041-5/+14
| | | | | | | | | | Signed-off-by: Francois Perrad <francois.perrad@gadz.org> [Thomas: - add comment in scancpan about the version dependency, suggested by Yann E. Morin. - add comment in perl.mk about the need to sync any version change with scancpan, also suggested by Yann E. Morin.] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* scanpypi: change hash file commentsGravatar Yegor Yefremov2016-07-041-3/+2
| | | | | | | | | | sha256 checksum will be computed locally either by scanpypi at package creation or by hand by package updates. Define this checksum as 'computed locally' so that one doesn't need to change this comment by package updates. Also put comments for both md5 and sha256 in one line. Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/download: don't over-remove files from git archivesGravatar Yann E. MORIN2016-07-041-3/+3
| | | | | | | | | | | | | | | | | When we now manually create git archives, we removed all .git-related files. However, we also exclude empty directories. This means that a directory which only had a .gitignore file is excluded from the archive. Fixes: http://autobuild.buildroot.org/results/2aa/2aa8954311f009988880d27b6e48af91bc74c346/ http://autobuild.buildroot.org/results/b45/b45cceea99b9860ccf1c925eeda498a823b30903/ http://autobuild.buildroot.org/results/5ae/5ae336052fd32057d9631649279e142a81f5651f/ http://autobuild.buildroot.org/results/5fc/5fc3abf4a1aea677f576e16c49253d00720a8bef/ Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* scanpypi: fix help text handlingGravatar Yegor Yefremov2016-07-041-0/+5
| | | | | | | Make sure a help text is terminated with a full stop. Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* pkg-cmake.mk: export the fortran compiler path in the CMake toolchain fileGravatar Samuel Martin2016-07-031-0/+4
| | | | | | | | | Since the fortran support is conditional, only enable it when needed. Signed-off-by: Samuel Martin <s.martin49@gmail.com> Signed-off-by: Vicente Olivert Riera <Vincent.Riera@imgtec.com> Cc: Benjamin Kamath <bkamath@spaceflight.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* core/pkg-infra: download git submodules if the package wants themGravatar Yann E. MORIN2016-07-021-3/+4
| | | | | | | | | | | | | | | Add a new package variable that packages can set to specify that they need git submodules. Only accept this option if the download method is git, as we can not get submodules via an http download (via wget). Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Aleksandar Simeonov <aleksandar@barix.com> Tested-by: Matt Weber <matt@thewebers.ws> Reviewed-by: Matt Weber <matt@thewebers.ws> Tested-By: Nicolas Cavallari <nicolas.cavallari@green-communications.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* support/download/git: add support for submodulesGravatar Yann E. MORIN2016-07-021-3/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | Some git repositories may be split into a master repository and submodules. Up until now, we did not have support for submodules, because we were using bare clones, in which it is not possible to update the list of submodules. Now that we are using plain clones with a working copy, we can retrieve the submdoules. Add an option to the git download helper to kick the update of submodules, so that they are only fetched for those packages that require them. Also document the existing -q option at the same time. Submodules have a .git file at their root, which contains the path to the real .git directory of the master repository. Since we remove it, there is no point in keeping those .git files either. Note: this is currently unused, but will be enabled with the follow-up patch that adds the necessary parts in the pkg-generic and pkg-download infrastructures. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Tested-by: Matt Weber <matt@thewebers.ws> Reviewed-by: Matt Weber <matt@thewebers.ws> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* support/download/git: do not use git archive, handle it manuallyGravatar Yann E. MORIN2016-07-021-3/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We currently use git-archive to generate the tarball. This is all handy and dandy, but git-archive does not support submodules. In the follow-up patch, we're going to handle submodules, so we would not be able to use git-archive. Instead, we manually generate the archive: - extract the tree to the requested cset, - get the date of the commit to store in the archive, - store only numeric owners, - store owner and group as 0 (zero, although any arbitrary value would have been fine, as long as it's a constant), - sort the files to store in the archive. We also get rid of the .git directory, because there is no reason to keep it in the context of Buildroot. Some people would love to keep it so as to speed up later downloads when updating a package, but that is not really doable. For example: - use current Buildroot - it would need foo-12345, so do a clone and keep the .git in the generated tarball - update Buildroot - it would need foo-98765 For that second clone, how could we know we would have to first extract foo-12345 ? So, the .git in the archive is pretty much useless for Buildroot. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Tested-by: Matt Weber <matt@thewebers.ws> Reviewed-by: Matt Weber <matt@thewebers.ws> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* support/download/git: do not use bare clonesGravatar Yann E. MORIN2016-07-021-3/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, we are using bare clones, so as to minimise the disk usage, most notably for largeish repositories such as the one for the Linux kernel, which can go beyond the 1GiB barrier. However, this precludes updating (and thus using) the submodules, if any, of the repositories, as a working copy is required to use submodules (becaue we need to know the list of submodules, where to find them, where to clone them, what cset to checkout, and all those is dependent upon the checked out cset of the father repository). Switch to using /plain/ clones with a working copy. This means that the extra refs used by some forges (like pull-requests for Github, or changes for gerrit...) are no longer fetched as part of the clone, because git does not offer to do a mirror clone when there is a working copy. Instead, we have to fetch those special refs by hand. Since there is no easy solution to know whether the cset the user asked for is such a special ref or not, we just try to always fetch the cset requested by the user; if this fails, we assume that this is not a special ref (most probably, it is a sha1) and we defer the check to the archive creation, which would fail if the requested cset is missing anyway. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Tested-by: Matt Weber <matt@thewebers.ws> Reviewed-by: Matt Weber <matt@thewebers.ws> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* scanpypi: rework runtime dependency handlingGravatar Yegor Yefremov2016-07-011-3/+1
| | | | | | | | | | | | | | In most cases Python's package dependencies found in setup.py are runtime dependencies and hence don't need to be mentioned in *.mk file. Also add '# runtime' tag to select statements in Config.in. __create_mk_requirements() itself is left for future uses (cffi backend handling etc.). Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* legal-info: explicitly state how patches are licensedGravatar Luca Ceresoli2016-06-241-1/+2
| | | | | | | | | | Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net> Acked-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> [yann.morin.1998@free.fr: slightly tweak after Arnout's review] Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr> Cc: Arnout Vandecappelle <arnout@mind.be> Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* core/legal-info: also save patchesGravatar Yann E. MORIN2016-06-241-4/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, the legal-info infra only saves the source archive of a package. However, that's not enough as we may apply some patches on packages sources. We do suggest users to also redistribute the Buildroot sources as part of their compliance distribution, so the patches bundled in Buildroot would indeed be included in the compliance distribution. However, that's still not enough, since we may download some patches, or the user may use a global patch directory. Patches in there might not end up in the compliance distribution, and there are risks of non-conformity. So, always include patches alongside the source archive. To ensure reproducibility, we also generate a series file, so patches can be re-applied in the correct order. We get the list of patches to include from the list of patches that were applied by the package infrastructure (via the apply-patches support script). So, we need to get packages properly extracted and patched before we can save their legal-info, not just in the case they define _LICENSE_FILES. Update the legal-info header accordingly. Note: this means that, when a package is not patched and defines no LICENSE_FILES, we will extract and patch it for nothing. There is no easy way to know whether we have to patch a package or not. We can only either duplicate the logic to detect patches (bad) or rely on the infra actually patching the package. Also, a vast majority of packages are either patched, or define _LICENSE_FILES, so it is best and easiest to always extract and patch them prior to legal-info. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Luca Ceresoli <luca@lucaceresoli.net> Tested-by: Luca Ceresoli <luca@lucaceresoli.net> Reviewed-by: Luca Ceresoli <luca@lucaceresoli.net> Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/apply-patches: bail-out on duplicate patch basenamesGravatar Yann E. MORIN2016-06-241-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Patches we save can come from various locations: - bundled with Buildroot - downloaded - from one or more global-patch-dir It is possible that two patches lying into different locations have the same basename, like so (first is bundled, second is from an hypothetical global-patch-dir): package/foo/0001-fix-Makefile.patch /path/to/my/patches/foo/0001-fix-Makefile.patch In that case, when running legal-info, we'd save only the second patch, overwriting the first. That would be problematic, because: - either the second patch depends on the first, and thus would no longer apply (this is easy to detect, though), - or the second patch does not depend on the first, and the compliance delivery will not be complete (this is much harder to detect). We fix that by checking that no two patches have the same same basename. If we find that the basename of the patch to be applied collides with that of a previously applied patch, we error out and report the duplicate. The unfortunate side-effect is that existing setups will now break in that situation, but that's a minor, corner-case issue that is easily fixed. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Luca Ceresoli <luca@lucaceresoli.net> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> [Thomas: adjust coding style, fix minor typos in the commit log.] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* core/apply-patches: store full path of applied patchesGravatar Yann E. MORIN2016-06-241-3/+8
| | | | | | | | | | | | | | | | | | | | | | | Currently, we only store the filename of the applied patches. However, we are soon to want to install those patches in the legal-info directory, so we'll have to know where those patches come from. Instead of duplicating the logic to find the patches (bundled, downloaded, from a global patch dir...), just store the full path to each of those patches so we can retrieve them more easily later on. Also always create the list-file, even if empty, so that we need not test for its existence before reading it. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Luca Ceresoli <luca@lucaceresoli.net> Reviewed-by: Luca Ceresoli <luca@lucaceresoli.net> [Tested only with patches in the Buildroot sources] Tested-by: Luca Ceresoli <luca@lucaceresoli.net> Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> [Thomas: used $PWD instead of $(pwd), as suggested by Arnout.] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* scanpypi: new utilityGravatar Denis THULIN2016-06-111-0/+650
| | | | | | | | | | | | A utility for creating python package from the python package index. It fetches packages info from http://pypi.python.org and generates corresponding packages files. Signed-off-by: Denis THULIN <denis.thulin@openwide.fr> Tested-by: Carlos Santos <casantos@datacom.ind.br> Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com> [Thomas: minor tweaks.] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* build: cleanup remaining references to BR2_STRIP_sstripGravatar Julien Floret2016-06-101-4/+0
| | | | | | | | sstrip option has been removed in commit 106f5b8c5093 ("build: remove deprecated sstrip option"). Signed-off-by: Julien Floret <julien.floret@6wind.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/gnuconfig: bump version to 2016-05-15Gravatar Bernd Kuhls2016-06-053-249/+169
| | | | | Signed-off-by: Bernd Kuhls <bernd.kuhls@t-online.de> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* Merge branch 'next'Gravatar Peter Korsgaard2016-06-012-0/+59
|\ | | | | | | Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
| * python/python3: globalize *.pyc files compilationGravatar Yegor Yefremov2016-05-261-0/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, each python package (be it the python interpreter package itself or external python modules) is responsible for compiling its .py into .pyc files. Unfortunately, this is not ideal as some packages only install .py files without compiling them into .pyc files. In this case, if the Buildroot configuration specifies to keep only the .pyc files, the .py files are removed and lost. To address this, this commit changes the logic by making the compilation of .pyc files a global operation: the python interpreter packages register a target finalize hook that is in charge of compiling all installed .py files. The *.pyc generation on a per package basis is disabled in the python-package infrastructure by passing the "--no-compile" option to setup.py. The *.pyc generation for the Python interpreter internal modules is disabled through --disable-pyc-build configure option. A small helper script is used to perform the compilation, the purpose of this script is to abort the compilation process if one of the .py file cannot be compiled. It has been provided by Samuel Martin and integrated into this commit. Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com> Cc: Samuel Martin <s.martin49@gmail.com> [Thomas: - rework for python 3.5 - integrate Samuel proposal that allows to detect compilation failures.] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Reviewed-by: Samuel Martin <s.martin49@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
| * support/scripts: add helper to hardlink-or-copyGravatar Yann E. MORIN2016-05-111-0/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When preparing the legal-info, the source archives are copied in the legal-info/ output directory. When the archives are big, it can take quite a bit of time and unnecessarily uses disk space. When the legal-info output directory is on the same filesystem as the BR2_DL_DIR, we can easily reduce copy time and disk usage by just using hardlins instead of copying. However, the BR2_DL_DIR may be on a different filesystem, so we must fallback to copying in this case Introduce a helper script that copies a source file into a destination directory, by first attempting to hard-link, and falling back to a plain copy in case the hardlink fails. In case the destination already exists, it is forcibly removed first, to avoid clobering any existing target file (and especially any hardlink to it), since cp -f does not remove the destination file, but clobbers it. In some situations, it will be necessary that the destination file is named differently than the source, so if a third argument is specified, it is treated as the basename of the destination file. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Luca Ceresoli <luca@lucaceresoli.net> Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* | Update for 2016.052016.05Gravatar Peter Korsgaard2016-05-311-1/+1
| | | | | | | | Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* | support/scripts: fix graph-build-time help textGravatar Thomas Petazzoni2016-05-241-1/+1
|/ | | | | | | | | | | | The graph-build-time help text currently looks like this: usage: graph-build-time [-h] [--type GRAPH_TYPE] [--order GRAPH_ORDER] [--alternate-colors] [--input OUTPUT] --output OUTPUT Obviously, naming the parameter for --input as OUTPUT is not a very good idea, so this commit fixes that to name it "INPUT", as expected. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/scripts/check-host-rpath: also check HOST_DIR/{bin, sbin}Gravatar Samuel Martin2016-04-211-1/+1
| | | | | | | | At least syslinux is installing stuff in HOST_DIR/sbin. Cc: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Samuel Martin <s.martin49@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/scripts: fix graph-depends when run with python3Gravatar Samuel Martin2016-04-181-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Make graph-depends script opening the output file in text mode since only ascii characters will be written. This change fixes the following error occuring when the default host python interpreter is python3: make: Entering directory '/opt/buildroot' Getting targets Getting dependencies for ['toolchain-external', 'toolchain', 'busybox', ...] Getting dependencies for ['host-python3', 'host-pkgconf', 'host-gettext', ...] Getting dependencies for ['host-libxml2', 'host-swig', 'host-m4', ...] Getting version for ['toolchain-external', 'toolchain', 'busybox', ...] Traceback (most recent call last): File "/opt/buildroot/support/scripts/graph-depends", line 425, in <module> outfile.write("digraph G {\n") TypeError: a bytes-like object is required, not 'str' Makefile:807: recipe for target 'graph-depends' failed make[1]: *** [graph-depends] Error 1 Makefile:84: recipe for target '_all' failed make: *** [_all] Error 2 make: Leaving directory '/opt/buildroot' While with python2, adding 'b' to the openning mode has no effect on Linux (c.f. [2]), the above error is expected with python3 (c.f. [1]). Therefore, just open the outfile in default (i.e. text) mode. [1] https://docs.python.org/3/library/functions.html#open [2] https://docs.python.org/2/library/functions.html#open Signed-off-by: Samuel Martin <s.martin49@gmail.com> Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* scancpan: use recommend & test flags only at first levelGravatar Francois Perrad2016-03-151-6/+6
| | | | | | | | Currently, these flags are recursively propagated. This behavior is not expected by users, because it can cause dependencies explosively. Signed-off-by: Francois Perrad <francois.perrad@gadz.org> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* scancpan: handle recommended dependencies as optional packagesGravatar Francois Perrad2016-03-151-3/+22
| | | | | | | | | | | | | | Currently, without the flag -recommend, scancpan takes as dependency only one which has the relationship "requires"; this mode works fine. And, with the flag -recommend, scancpan takes all ones (ie. with relationship "requires" or "recommends") in the same way; this mode never works fine, because it is too simplistic. With this commit, the "not required" dependencies are handled as optional BR package or skipped if a cyclic dependency is detected. Signed-off-by: Francois Perrad <francois.perrad@gadz.org> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* Update for 2016.022016.02Gravatar Peter Korsgaard2016-03-011-1/+1
| | | | Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* support/download: really, really make git archives reproducibleGravatar Yann E. MORIN2016-02-271-1/+1
| | | | | | | | | | | | | | | The way we use it, gzip will store the current time in the header, which leads to unreproducible archives. Fix that by telling gzip to not store the name and date of the file it compresses, with the -n option. Since it compresses its stdin, there was already no filename stored; now there's even no date stored. Note: gzip has had -n since at least 1.2.4, released in 1993, so virtually every gzip out there nowadays has it. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* graph-depends: add support for excluding host packagesGravatar Thomas Petazzoni2016-02-081-2/+8
| | | | | | | | | | | | | | Just like the --stop-on and --exclude options allow to stop on or exclude virtual packages from the list by passing the "virtual" magic value, this commit extends the graph-depends logic to support a "host" magic value for --stop-on and --exclude. This will allow to draw the graph by stopping on host packages, or by excluding host packages. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> [Thomas: minor code beautification suggested by Yann E. Morin.] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* graph-depends: fix handling of "virtual" in exclude_listGravatar Thomas Petazzoni2016-02-081-4/+3
| | | | | | | | | | | | | | | | | | | | | The condition to determine if a virtual package should be excluded from the list due to "virtual" being passed in --exclude is under a loop iterating over each entry of the exclude_list, but it doesn't use the iterator of this list. Indeed, the condition contains: "virtual" in exclude_list which checks automatically if "virtual" was passed in the list. Due to this, there is no need for this check to be within the "for p in exclude_list" iteration. This commit fixes that by moving the check outside of the loop. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/graph-depends: teach it to only check dependenciesGravatar Yann E. MORIN2016-02-071-0/+10
| | | | | | | | | | Add an option to graph-depends to only do the dependency checks and not generate the dot program. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Samuel Martin <s.martin49@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/graph-depends: detect circular dependenciesGravatar Yann E. MORIN2016-02-071-0/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, if there is a circular dependency in the packages, the graph-depends script just errors out with a Python RuntimeError which is not caught, resulting in a very-long backtrace which does not provide any hint as what the real issue is (even if "RuntimeError: maximum recursion depth exceeded" is a pretty good hint at it). We fix that by recursing the dependency chain of each package, until we either end up with a package with no dependency, or with a package already seen along the current dependency chain. We need to introduce a new function, check_circular_deps(), because we can't re-use the existing ones: - remove_mandatory_deps() does not iterate, - remove_transitive_deps() does iterate, but we do not call it for the top-level package if it is not 'all' - it does not make sense to use those functions anyway, as they were not designed to _check_ but to _act_ on the dependency chain. Since we've had time-related issues in the past, we do not want to introduce yet another time-hog, so here are timings with the circular dependency check: $ time python -m cProfile -s cumtime support/scripts/graph-depends [...] 28352654 function calls (20323050 primitive calls) in 87.292 seconds Ordered by: cumulative time ncalls tottime percall cumtime percall filename:lineno(function) 1 0.012 0.012 87.292 87.292 graph-depends:24(<module>) 21 0.000 0.000 73.685 3.509 subprocess.py:473(_eintr_retry_call) 7 0.000 0.000 73.655 10.522 subprocess.py:768(communicate) 7 73.653 10.522 73.653 10.522 {method 'read' of 'file' objects} 5/1 0.027 0.005 43.488 43.488 graph-depends:164(get_all_depends) 5 0.003 0.001 43.458 8.692 graph-depends:135(get_depends) 1 0.001 0.001 25.712 25.712 graph-depends:98(get_version) 1 0.001 0.001 13.457 13.457 graph-depends:337(remove_extra_deps) 1717 1.672 0.001 13.050 0.008 graph-depends:290(remove_transitive_deps) 9784086/2672326 5.079 0.000 11.363 0.000 graph-depends:274(is_dep) 2883343/1980154 2.650 0.000 6.942 0.000 graph-depends:262(is_dep_uncached) 1 0.000 0.000 4.529 4.529 graph-depends:121(get_targets) 2883343 1.123 0.000 1.851 0.000 graph-depends:246(is_dep_cache_insert) 9784086 1.783 0.000 1.783 0.000 graph-depends:255(is_dep_cache_lookup) 2881580 0.728 0.000 0.728 0.000 {method 'update' of 'dict' objects} 1 0.001 0.001 0.405 0.405 graph-depends:311(check_circular_deps) 12264/1717 0.290 0.000 0.404 0.000 graph-depends:312(recurse) [...] real 1m27.371s user 1m15.075s sys 0m12.673s The cumulative time spent in check_circular_deps is just below 0.5s, which is largely less than 1% of the total run time. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Samuel Martin <s.martin49@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/graph-depends: add option to specify output fileGravatar Yann E. MORIN2016-02-071-6/+13
| | | | | | | | | | | | | | | | | | | | | Currently, graph-depends outputs the dotfile program to stdout, and uses stderr to trace the dependencies it is currently looking for. Redirection was done because the output was directly piped into the dot program to generate the final PDF/SVG/... dependency graph, but that meant that an error in the graph-depends script was never caught (because shell pipes only return the final command exit status, and an empty dot program is perfectly valid so dot would not complain). Add an option to tell graph-depends where to store the generated dot program, and keep stdout as the default if not specified. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Samuel Martin <s.martin49@gmail.com> [Thomas: rename metavar from DOT_FILE to OUT_FILE for consistency with the rest of the new option naming.] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/scripts: add size-stats-compare scriptGravatar Thomas De Schampheleire2016-02-071-0/+127
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Leverage the CSV files produces by size-stats (make graph-size) to allow for a comparison of rootfs size between two different buildroot compilations. The script takes the file-size CSV files of two compilations as input, and produces a textual report of the differences per package. Using the -d/--detail flag, the report will show the file size changes instead of package size changes. The -t/--threshold option allows to ignore file size differences smaller or equal than the given threshold (in bytes). Example output is: Size difference per package (bytes), threshold = 0 -------------------------------------------------------------------------------- -8192 busybox 228572 added dmalloc 301584 added jq -------------------------------------------------------------------------------- 521964 TOTAL or with detailed view: Size difference per file (bytes), threshold = 0 -------------------------------------------------------------------------------- -8192 bin/busybox 18152 added usr/bin/jq 39252 added usr/bin/dmalloc 46968 added usr/lib/libdmalloc.so 47288 added usr/lib/libdmallocxx.so 47316 added usr/lib/libdmallocth.so 47748 added usr/lib/libdmallocthcxx.so 283432 added usr/lib/libjq.so.1.0.4 -------------------------------------------------------------------------------- 521964 TOTAL Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>