Last modified: 2014-08-17 23:48:41 UTC
Reported on the #wikimedia-qa IRC channel, and I was able to repro locally. When a user provisions the "browsertests" role for mediawiki-vagrant, the obsolete mediawiki-selenium is deployed, rather than the newer (and expected) mediawiki_selenium Rubygem (note the underscore). Repro steps: 1) Start with a fresh mediawiki-vagrant install (i.e. git clone the project). 2) cd to new "vagrant" sub-directory. 3) vagrant up 4) vagrant enable-role browsertests 5) vagrant provision (this step produces the error message shown below) As near as I can tell, the error shown below does not actually prevent the "browsertests" role from working as expected; I am still able to run browser tests from the /srv/browsertests/tests/browser directory on the VM. Error message: notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: Could not locate Gemfile err: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: change from notrun to 0 failed: bundle install --path /home/vagrant/.gem returned 10 instead of one of [0] at /tmp/vagrant-puppet-1/modules-0/browsertests/manifests/init.pp:85
Complete messaging from the "vagrant provision" command is as follows: Jeffrey-Hall:vagrant jeffreyhall$ vagrant enable-role browsertests Ok. Run 'vagrant provision' to apply your changes. Jeffrey-Hall:vagrant jeffreyhall$ vagrant provision [default] Running provisioner: puppet... Running Puppet with site.pp... info: Applying configuration version '1392241885.c958ae3' info: mount[files]: allowing mediawiki-vagrant access notice: /Stage[main]/Browsertests/Env::Var[MEDIAWIKI_URL]/Env::Profile[set_mediawiki_url]/File[/etc/profile.d/puppet-managed/set_mediawiki_url.sh]/ensure: created notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: Could not locate Gemfile err: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: change from notrun to 0 failed: bundle install --path /home/vagrant/.gem returned 10 instead of one of [0] at /tmp/vagrant-puppet-1/modules-0/browsertests/manifests/init.pp:85 notice: Finished catalog run in 3.76 seconds The following SSH command responded with a non-zero exit status. Vagrant assumes that this means the command failed! FACTER_fqdn='mediawiki-vagrant.dev' FACTER_forwarded_port='8080' FACTER_shared_apt_cache='/vagrant/apt-cache/' FACTER_provider_name='virtualbox' FACTER_provider_version='4.3.6' puppet apply --templatedir /vagrant/puppet/templates --verbose --config_version /vagrant/puppet/extra/config-version --fileserverconfig /vagrant/puppet/extra/fileserver.conf --logdest /vagrant/logs/puppet/puppet.c958ae375.log --logdest console --modulepath '/tmp/vagrant-puppet-1/modules-0' --manifestdir /tmp/vagrant-puppet-1/manifests --detailed-exitcodes /tmp/vagrant-puppet-1/manifests/site.pp || [ $? -eq 2 ] Stdout from the command: info: Applying configuration version '1392241885.c958ae3' info: mount[files]: allowing mediawiki-vagrant access notice: /Stage[main]/Browsertests/Env::Var[MEDIAWIKI_URL]/Env::Profile[set_mediawiki_url]/File[/etc/profile.d/puppet-managed/set_mediawiki_url.sh]/ensure: created notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: Could not locate Gemfile err: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: change from notrun to 0 failed: bundle install --path /home/vagrant/.gem returned 10 instead of one of [0] at /tmp/vagrant-puppet-1/modules-0/browsertests/manifests/init.pp:85 notice: Finished catalog run in 3.76 seconds
Change 113988 had a related patch set uploaded by EBernhardson: Repoint bundle install cwd https://gerrit.wikimedia.org/r/113988
Change 113988 merged by jenkins-bot: Repoint bundle install cwd https://gerrit.wikimedia.org/r/113988
Now getting: jdforrester@local:~/Documents/git/vagrant$ vagrant enable-role browsertests && vagrant provision Ok. Run 'vagrant provision' to apply your changes. [default] Running provisioner: puppet... Running Puppet with site.pp... info: Applying configuration version '1394576869.c7f1990' info: mount[files]: allowing mediawiki-vagrant access notice: /Stage[main]/Browsertests/Env::Var[MEDIAWIKI_URL]/Env::Profile[set_mediawiki_url]/File[/etc/profile.d/puppet-managed/set_mediawiki_url.sh]/ensure: created notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: /usr/lib/ruby/1.9.1/fileutils.rb:247:in `mkdir': Permission denied - /home/vagrant/.gem/ruby (Errno::EACCES) notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/1.9.1/fileutils.rb:247:in `fu_mkdir' notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/1.9.1/fileutils.rb:221:in `block (2 levels) in mkdir_p' notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/1.9.1/fileutils.rb:219:in `reverse_each' notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/1.9.1/fileutils.rb:219:in `block in mkdir_p' notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/1.9.1/fileutils.rb:205:in `each' notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/1.9.1/fileutils.rb:205:in `mkdir_p' notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/1.9.1/pathname.rb:514:in `mkpath' notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/vendor_ruby/bundler/installer.rb:15:in `run' notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/vendor_ruby/bundler/installer.rb:8:in `install' notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/vendor_ruby/bundler/cli.rb:222:in `install' notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/vendor_ruby/bundler/vendor/thor/task.rb:22:in `run' notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/vendor_ruby/bundler/vendor/thor/invocation.rb:118:in `invoke_task' notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/vendor_ruby/bundler/vendor/thor.rb:246:in `dispatch' notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/vendor_ruby/bundler/vendor/thor/base.rb:389:in `start' notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/bin/bundle:13:in `<main>' err: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: change from notrun to 0 failed: bundle install --path /home/vagrant/.gem returned 1 instead of one of [0] at /tmp/vagrant-puppet/modules-0/browsertests/manifests/init.pp:85 notice: Finished catalog run in 4.81 seconds The following SSH command responded with a non-zero exit status. Vagrant assumes that this means the command failed! FACTER_fqdn='mediawiki-vagrant.dev' FACTER_forwarded_port='8080' FACTER_shared_apt_cache='/vagrant/apt-cache/' FACTER_provider_name='virtualbox' FACTER_provider_version='4.2.18' puppet apply --templatedir /vagrant/puppet/templates --verbose --config_version /vagrant/puppet/extra/config-version --fileserverconfig /vagrant/puppet/extra/fileserver.conf --logdest /vagrant/logs/puppet/puppet.c7f199012.log --logdest console --modulepath '/etc/puppet/modules:/tmp/vagrant-puppet/modules-0' --manifestdir /tmp/vagrant-puppet/manifests --detailed-exitcodes /tmp/vagrant-puppet/manifests/site.pp || [ $? -eq 2 ] Stdout from the command: info: Applying configuration version '1394576869.c7f1990' info: mount[files]: allowing mediawiki-vagrant access notice: /Stage[main]/Browsertests/Env::Var[MEDIAWIKI_URL]/Env::Profile[set_mediawiki_url]/File[/etc/profile.d/puppet-managed/set_mediawiki_url.sh]/ensure: created notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: /usr/lib/ruby/1.9.1/fileutils.rb:247:in `mkdir': Permission denied - /home/vagrant/.gem/ruby (Errno::EACCES) notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/1.9.1/fileutils.rb:247:in `fu_mkdir' notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/1.9.1/fileutils.rb:221:in `block (2 levels) in mkdir_p' notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/1.9.1/fileutils.rb:219:in `reverse_each' notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/1.9.1/fileutils.rb:219:in `block in mkdir_p' notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/1.9.1/fileutils.rb:205:in `each' notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/1.9.1/fileutils.rb:205:in `mkdir_p' notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/1.9.1/pathname.rb:514:in `mkpath' notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/vendor_ruby/bundler/installer.rb:15:in `run' notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/vendor_ruby/bundler/installer.rb:8:in `install' notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/vendor_ruby/bundler/cli.rb:222:in `install' notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/vendor_ruby/bundler/vendor/thor/task.rb:22:in `run' notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/vendor_ruby/bundler/vendor/thor/invocation.rb:118:in `invoke_task' notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/vendor_ruby/bundler/vendor/thor.rb:246:in `dispatch' notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/vendor_ruby/bundler/vendor/thor/base.rb:389:in `start' notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/bin/bundle:13:in `<main>' err: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: change from notrun to 0 failed: bundle install --path /home/vagrant/.gem returned 1 instead of one of [0] at /tmp/vagrant-puppet/modules-0/browsertests/manifests/init.pp:85 notice: Finished catalog run in 4.81 seconds Stderr from the command: jdforrester@local:~/Documents/git/vagrant$
*** Bug 63990 has been marked as a duplicate of this bug. ***
Change 126207 had a related patch set uploaded by Ori.livneh: Make sure ~vagrant/.gem is owned by vagrant:vagrant https://gerrit.wikimedia.org/r/126207
James or Stefan, could you verify that the patch fixes things?
Change 126207 merged by jenkins-bot: Make sure ~vagrant/.gem is owned by vagrant:vagrant https://gerrit.wikimedia.org/r/126207
I believe I'm running into the same issue, even with proper ownership and permissions on ~vagrant/.gem. A manual execution of `bundle install` reveals a failed compilation of the ffi gem. At first I thought this might be a missing dependency but the issue persists after installing libffi—and looking into the gem source it appears to be bundled with its own libffi so that's definitely not it. I'll do some more digging. Output of `bundle install --path /home/vagrant/.gem` in /srv/browsertests/tests/browser: Fetching source index for https://rubygems.org/ Using builder (3.2.2) Installing ffi (1.9.3) with native extensions /usr/lib/ruby/1.9.1/rubygems/installer.rb:552:in `rescue in block in build_extensions': ERROR: Failed to build gem native extension. (Gem::Installer::ExtensionBuildError) /usr/bin/ruby1.9.1 extconf.rb Gem files will remain installed in /home/vagrant/.gem/ruby/1.9.1/gems/ffi-1.9.3 for inspection. Results logged to /home/vagrant/.gem/ruby/1.9.1/gems/ffi-1.9.3/ext/ffi_c/gem_make.out (... useless rubygems backtrace)
The failure I was having appears to have been an issue with the older packaged version of Bundler that is installed by `puppet/modules/browsertests/manifests/init.pp`. After removing it (`apt-get remove --purge ruby-bundler`) and installing the latest RubyGems version, I can `bundle install` without a problem. A short-term workaround to this issue would be to make sure the Puppet manifest installs Bundler using the "gem" provider. However, there is a separate issue at the moment that makes this workaround a bit messy: The GEM_HOME environment variable is being set relative to the user's home directly (by our script in /etc/profile.d/puppet-managed/gem_home.sh) meaning that gems installed as root (which is what Puppet will do for a package resource) are not accessible to other users. In this case, the bundler gem installed via a Puppet package will not be accessible to the user trying to execute `bundle install` (the vagrant user). A broader solution (and maybe a more correct one in my opinion) would be to better isolate ruby dependencies via something like RVM or rbenv, where multiple ruby binaries and gems can be managed independently of system ones (see bug 50993) or even independently for users or specific applications. This will allow for more up-to-date versions of ruby and gems to be installed for specific roles without affecting others. I'll go down the second road (referencing bug 50993 as it seems more directly relevant) and, if I run into any major blockers, I'll come back and implement the short-term workaround.
[No patches left here - resetting bug report status]
The browsertests role was removed entirely in I6a02788a42a5dcd58dc03ce845fedfe791da3bcb. If there are still issues with the various roles that enable browser tests for specific extensions a new bug should be filed.
