Jenkins: Build-testing for every single commit

As of yesterday, I’ve drastically increased the number of testing machines in Jenkins (from two to 21!). This enables me to poll for changes in git every five minutes and automatically start a testing run for every single commit – regardless of it entering via Gerrit or not. (Note: This is in addition to the Gerrit trigger I’ve been using from day one on, not replacing it.)

Here’s an example from today:

https://galileo.mailstation.de/jenkins/job/arbor/67/

Apart from looking at Jenkins directly, you can join the #exherbo-bots channel where the jenkins-exherbo bot announces all test results. Here’s an example for that:

[03.01.2014 13:42:59] <irker657> Exherbo: pipping arbor:master * 76ec9309669d / packages/sys-apps/grep/grep-2.15.exheres-0 packages/sys-apps/grep/grep-2.16.exheres-0: grep: Bump to 2.16 http://tinyurl.com/pyfhbql
[03.01.2014 13:46:50] <jenkins-exherbo> Project arbor build #67:SUCCESS in 1 min 5 sec: https://galileo.mailstation.de/jenkins/job/arbor/67/

I hope you’ll find this useful and will actually look at the results since it will find out if you missed a dependency. And it potentially shows you automagic dependencies.

 

I am and have been working on quite a few F/OSS projects:
  • Exherbo (Nick: Philantrop)
  • Gentoo (Nick: Philantrop)
  • Calibre plugin iOS reader applications
  • Calibre plugin Marvin XD
  • chroot-manager
  • stuff on github
  • Lots of other projects
  • If you like my work, feel free to donate. 🙂

    News about Gerrit, Jenkins & friends

    Here are a few things I’ve updated on Gerrit, Jenkins and associated tools:

    1. Gerrit
      • The Gerrit bot, gerritwk23, has now support for !pl (and pl in a query). It’s just like zebrapig’s !pl.
      • I’ve updated Gerrit in mid-Decembre. Since then, you can switch to a newly designed “Change View” which you’ll find under your account’s settings in the “Change View” option. There’s also an associated “Diff View” option to set your preferred diff style if you switch to the new Change View.

    2. Jenkins

      • In its build logs, you’ll now find the mounts, the cave resolve command and the (likely) dependencies gathered from the installed package. Dependency information might not be 100% accurate so please take it with a grain of salt but it’s usually a good indicator.
        Look out for “**************************************************************” to find that information.

     

    If you have any ideas about what else to improve, please let me know.

    I am and have been working on quite a few F/OSS projects:
  • Exherbo (Nick: Philantrop)
  • Gentoo (Nick: Philantrop)
  • Calibre plugin iOS reader applications
  • Calibre plugin Marvin XD
  • chroot-manager
  • stuff on github
  • Lots of other projects
  • If you like my work, feel free to donate. 🙂

    Another good reason to use Gerrit: Jenkins is at your service

    If it’s not yet good enough for you that using Gerrit

    • is even easier and faster than using zebrapig (you just push your change and you’re done! And if you use git review it’s getting even easier.),
    • the review turn-around times are often better when using Gerrit than zebrapig,
    • discussing patches can be done both when you’re on IRC and on Gerrit,

    then here’s another good reason to use it: I’ve set up Jenkins for all repositories on Gerrit to build every change in a chroot (including sydbox-1, pinktrace-1 and docs).

    For now, all reports only go to me (and anyone else who wants to get an email with the result included) and Jenkins will not report back to Gerrit because the setup is quite fragile yet and I want to be sure it works properly before making it do more.

    No idea how to use Gerrit? Just read my earlier posts Gerrit Code Review for Exherbo and Gerrit for Core-Devs. Of course, you can ask me, too. 🙂

     

    I am and have been working on quite a few F/OSS projects:
  • Exherbo (Nick: Philantrop)
  • Gentoo (Nick: Philantrop)
  • Calibre plugin iOS reader applications
  • Calibre plugin Marvin XD
  • chroot-manager
  • stuff on github
  • Lots of other projects
  • If you like my work, feel free to donate. 🙂

    Exherbo’s Gerrit for 3rd party repositories

    Exherbo’s Gerrit has left its infancy and, thus, I can now add 3rd party repositories to our Gerrit installation. Please just let me know if you’d like that.

    • Requirements:
      • Your repository must use git.
      • You must have an account in Gerrit.
      • You must be willing to add a Gerrit user (github: Gerrit-Exherbo; BitBucket: gerrit-exherbo) to your repository as a contributor.  Here’s the public key. (You don’t need the public key for github or BitBucket; if you’re unsure, you most likely don’t need it.)
      • Your repository should be hosted on one of the big git hosting providers, e. g. github (preferred) or BitBucket. Let me know if your repository is hosted elsewhere.
      • Please let me know the push URI for your repository.

    Once I’ve set you up, you can immediately work on Gerrit patches for your repository. You are, of course, its sole owner and fully responsible for it.

    By default, Exherbo’s core devs (but nobody else unless you explicitly ask me to add someone) will be able to +2 Gerrit changes for your repository, too. This is meant to potentially help you keep your repository up to date (e. g. when mass changes occur). This is entirely optional, though, and you can choose to opt-out at any time if you don’t like it; you only need to let me know.

    I am and have been working on quite a few F/OSS projects:
  • Exherbo (Nick: Philantrop)
  • Gentoo (Nick: Philantrop)
  • Calibre plugin iOS reader applications
  • Calibre plugin Marvin XD
  • chroot-manager
  • stuff on github
  • Lots of other projects
  • If you like my work, feel free to donate. 🙂

    Gerrit installation updated to the 2.7 release, pesky “internal server error” fixed

    I’ve just updated the Gerrit installation to the 2.7 release and fixed the pesky “internal server error” that occurred when entering a user name or similar.

    I’ve also updated my posts on Gerrit and the FAQ-like post for core devs:

    Gerrit Code Review for Exherbo

    Gerrit for Core-Devs

     

     

    I am and have been working on quite a few F/OSS projects:
  • Exherbo (Nick: Philantrop)
  • Gentoo (Nick: Philantrop)
  • Calibre plugin iOS reader applications
  • Calibre plugin Marvin XD
  • chroot-manager
  • stuff on github
  • Lots of other projects
  • If you like my work, feel free to donate. 🙂

    Samba has no maintainer on Exherbo any more

    After having worked with Samba for almost 20 years, I'm giving up now.
    
    I have submitted patches upstream when Samba (and I :) ) was young,
    I've proof-read books on Samba and NTLM but where they're going, I
    can't follow them any more.
    
    - Samba 4.1.0 doesn't even build for me any more and I'm too
    frustrated to figure it out.
    
    - In Samba 4.1.0, SWAT (the configuration web interface) seems to have
    been removed. I might be missing something but if I'm not, that sucks.
    
    - Samba's "new" crappy build system WAF is just horrible and I simply
    don't understand it. I asked for help (and it was offered) but nothing
    was ever done.
    
    - New versions of Samba 4 try to build against already installed
    versions in the live filesystem and I don't understand WAF enough to
    fix that.
    
    I've just removed myself from BUGS_TO. If you want it, take it but if
    you do, do me a favour and do a better job than me and make sure
    you're weird enough to understand (and ideally like) WAF.
    I am and have been working on quite a few F/OSS projects:
  • Exherbo (Nick: Philantrop)
  • Gentoo (Nick: Philantrop)
  • Calibre plugin iOS reader applications
  • Calibre plugin Marvin XD
  • chroot-manager
  • stuff on github
  • Lots of other projects
  • If you like my work, feel free to donate. 🙂

    Gerrit for Core-Devs

    There have been a few questions and potential misunderstandings I’m going to try to address here. I’ll update this post if new stuff comes up.

    • Getting the foo Or: How do I become a Core-Dev on Gerrit?
      Simple: You register with your GitHub or Google account, then you ask any other core dev to add you to the group. At least Bo “zlin” Ørsted Andresen and I know how to do it and every member has the necessary privileges.
    • Reviewing Or: +1+1 != +2
      Every registered user can comment and/or -1/+1 a change. This is an indication said user has reviewed the patch and, ideally, tested it. It’s basically only an opinion, though.
      Only Core-Devs can +2 a change. This means they’ve reviewed and tested that change and are approving it. The change then will get merged and pushed.
      It’s important to remember, though, that +1’s do NOT add up. You can +1 a change a hundred times but it will still need one bold man (or woman) to +2 it.
      Important as well, don’t forget to press the “Submit” button if you want to get a change merged.
    • What about testing (or the lack of)?
      Everyone who +2’s a change at least (ideally, +1 would indicate you tested the thing, too!) is expected to have actually tested it. If you +2 it but didn’t test it, we’re going to slowly roast you over a scented candle. 
    • What’s the easiest way to test a patch?
      Use dev-scm/gertty.

      (Previsouly: I look at the patch, at the “Downloads” line, I click on “Patch” and then click on the small clipboard icon next to the git fetch comand. That adds said line to your clipboard. Next, I cd into the respective repository, paste the line we just copied and append “| git am”. That’s it, patch applied, now test it!)

    • Merging & pushing (no, not like that, you pervert!)
      Gerrit now pushes merged changes directly to our repositories. This means, once you +2 and submit a change (cf. first point), you push it.
    • Command line tricks or How do I…
      … most easily review a change?git review -d -t <change number, e. g. 202>

      … merge a boat-load of patches I reviewed and tested at once?

      On your branch with the changes applied: ssh -p 29418 <username>@galileo.mailstation.de gerrit review –code-review +2 –submit $(git rev-list gerrit/master..HEAD)

      … query for a specific change?

      ssh -p 29418 <username>@galileo.mailstation.de — gerrit query –comments change:212

      …  get a list of all open changes?

      ssh -p 29418 <username>@galileo.mailstation.de — gerrit query status:open (add a “project” parameter if you want only changes for a certain repository, e. g. “project:arbor”)

      … add a comment from the command line?

      ssh -p 29418 <username>@galileo.mailstation.de — gerrit review -m “\”At the very least this commit message should explain why we have to put PWD in PYTHONPATH which can’t be quoted properly… But other than that, seems sensible at least for now.\”” $changeid

    I am and have been working on quite a few F/OSS projects:
  • Exherbo (Nick: Philantrop)
  • Gentoo (Nick: Philantrop)
  • Calibre plugin iOS reader applications
  • Calibre plugin Marvin XD
  • chroot-manager
  • stuff on github
  • Lots of other projects
  • If you like my work, feel free to donate. 🙂

    Gerrit Code Review for Exherbo

    I’ve installed a Gerrit Code Review instance on my server for use with Exherbo.

    Gerrit is a code review tool and allows for

    • discussing patches and keeping the results for future reference
    • get notified by email about changes (if you want)
    • easily work on every Exherbo repository
    • contributors to get their repositories added to Gerrit as well (optional but strongly recommended)
    You’ll find an introduction to Gerrit here.
    Notes:
    • You need a GitHub or Google account. The email address you wish to use in Gerrit must be configured in your GitHub account.
    • You MUST use your real name for copyright reasons.
    • Gerrit replicates merged changes pretty much immediately. This sometimes/rarely fails on the first attempt for various reason. I’ve implemented a fallback that occurs every 5 minutes.

    How to use Gerrit:

    Initial setup in two easy steps:
    1. Go to https://galileo.mailstation.de/gerrit//, in the upper right corner, click “Sign in”.
    2. Authorize Gerrit with GitHub or Google.
     Cloning a repository:
    1. Click “Projects”, “List”, then choose a project.
    2. You should see several methods for cloning the project, the easiest way is to choose the ssh method.
    3. Clone, e. g. git clone ssh://<user name>@galileo.mailstation.de:29418/alip
    4. Install a hook that sets a Change-Id automatically (choose one of the two alternatives):

    scp -p -P 29418 <your username>@galileo.mailstation.de:hooks/commit-msg <local path to your repository>/.git/hooks/
    curl -o <local path to your repository>/.git/hooks/commit-msg https://galileo.mailstation.de/gerrit/tools/hooks/commit-msg

     

    chmod +x <local path to your repository>/.git/hooks/commit-msg

    Uploading changes:

    There are several methods to upload changes:
    1. Work on your copy of the repository, commit.
    2. Push your changes: git push origin HEAD:refs/for/master
    You can use git review for easily setting up your repository clone and submitting patches, too. It’s quite nice, use it!
    You might want to read the following posts as well:

     

     

     

    And if you just can’t get enough: All my posts about Exherbo’s Gerrit.

     

    Exherbo’s Patch Queue

    Today, on May, 5th 2012, our patch queue on Zebrapig the Ugly did the impossible; it achieved re-floration! 😉

     

    [05.05.2012 21:57:36] <Philantrop> !pl

    [05.05.2012 21:57:37] [Notice] -zebrapig to #exherbo- 0 patches in queue:

     

    I am and have been working on quite a few F/OSS projects:
  • Exherbo (Nick: Philantrop)
  • Gentoo (Nick: Philantrop)
  • Calibre plugin iOS reader applications
  • Calibre plugin Marvin XD
  • chroot-manager
  • stuff on github
  • Lots of other projects
  • If you like my work, feel free to donate. 🙂

    paste.pocoo.org is gone for good. Please re-submit your patches.

    paste.pocoo.org is gone for good. Please re-submit your patches.

    Here’s the list of affected patches:

    [28.04.2012 17:24:28] [Notice] -zebrapig- 13 matching patches in queue:
    [28.04.2012 17:24:28] [Notice] -zebrapig- http://paste.pocoo.org/raw/510003/ ::sydbox (submitted by worr 161 days and 17 hours ago): [PATCH] Fixed race condition in sydbox
    [28.04.2012 17:24:28] [Notice] -zebrapig- http://paste.pocoo.org/raw/579419/ ::kde (submitted by heirecka_ 17 days and 5 hours ago): [PATCH 1/2] Move stuff from qt-creator-scm to its exlib
    [28.04.2012 17:24:28] [Notice] -zebrapig- http://paste.pocoo.org/raw/582592/ ::arbor (submitted by SuperHeron 11 days and 20 hours ago): [PATCH] app-shells/bash-completion[=scm]: /etc/bash_completion.d is
    [28.04.2012 17:24:28] [Notice] -zebrapig- http://paste.pocoo.org/raw/582875/ ::x11 (submitted by moben 11 days and 8 hours ago): [PATCH 1/2] mesa multilib: Use multiunpack exparam
    [28.04.2012 17:24:30] [Notice] -zebrapig- http://paste.pocoo.org/raw/583000/ ::arbor (submitted by keruspe 11 days and 1 hour ago): [PATCH] llvm: allow gcc 4.7.0
    [28.04.2012 17:24:32] [Notice] -zebrapig- http://paste.pocoo.org/raw/584582/ ::mawww (submitted by heirecka 8 days and 3 hours ago): [PATCH] Fix building of espeak
    [28.04.2012 17:24:34] [Notice] -zebrapig- http://paste.pocoo.org/raw/585239/ ::arbor (submitted by SuperHeron 7 days and 4 minutes ago): [PATCH] dev-lang/python: Add .pc file into alternatives
    [28.04.2012 17:24:36] [Notice] -zebrapig- http://paste.pocoo.org/raw/585373/ ::anderslm (submitted by Philantrop 6 days and 18 hours ago): [PATCH] dev-libs/dotconf: ::anderslm -> ::media
    [28.04.2012 17:24:38] [Notice] -zebrapig- http://paste.pocoo.org/raw/586847/ ::kde (submitted by Apetrini 3 days and 13 hours ago): [PATCH] networkmanagement: bump to 0.9.0.1-r1, fix dep
    [28.04.2012 17:24:40] [Notice] -zebrapig- http://paste.pocoo.org/raw/588002/ ::ingmar (submitted by nakamuray 1 day and 3 minutes ago): [PATCH 1/3] notmuch: version bump to 0.12
    [28.04.2012 17:24:42] [Notice] -zebrapig- http://paste.pocoo.org/raw/588028/ ::arbor (submitted by keruspe 23 hours and 6 minutes ago): [PATCH 1/3] autoconf: bump to 2.69
    [28.04.2012 17:24:44] [Notice] -zebrapig- http://paste.pocoo.org/raw/588037/ ::x11 (submitted by keruspe 22 hours and 47 minutes ago): [PATCH] xkeyboard-config-scm: fix build
    [28.04.2012 17:24:46] [Notice] -zebrapig- http://paste.pocoo.org/raw/588130/ ::kde (submitted by heirecka_ 19 hours and 56 minutes ago): [PATCH] Version bump to akonadi-1.7.2

    I am and have been working on quite a few F/OSS projects:
  • Exherbo (Nick: Philantrop)
  • Gentoo (Nick: Philantrop)
  • Calibre plugin iOS reader applications
  • Calibre plugin Marvin XD
  • chroot-manager
  • stuff on github
  • Lots of other projects
  • If you like my work, feel free to donate. 🙂