We believe that the best way to build software is to do it in close collaboration with the people who use it. We invite you to submit your ideas using the form below. Please be sure to include the problem for which you are solving and the benefits of implementing the idea.
We do our best to implement as many Ideas as we can. Our Product team will evaluate all submitted ideas in a timely manner and will disposition each into one of the following categories: will integrate into the product roadmap, further research is needed, unlikely to implement.
Thanks for collaborating with us!
If we (Chef) could take the time to invest in addressing some long-standing issues with Test Kitchen, it will make using our kit much better. I'm going to highlight two issues here, however the "idea" is to invest some more product cycles into making Test Kitchen easier to use and more robust:
Here are the two issues that are currently affecting more than one of my customers:
# Managing Chef versions during kitchen testing if the AMI already has Chef baked in
kitchen provisioner install strategy options are: always/once/skip
https://docs.chef.io/workstation/config_yml_kitchen/#new-provisioner-settings
This is super annoying when we have Chef14 (or really any version that’s not latest stable) baked into an AMI and then we want to have a kitchen suite that always tests against latest stable Chef.
The current workaround is to set the product_name: chef-workstation, which gets usually (but not always) has the latest stable bundled in. It’s kludgy and is a pain to explain yet-another workaround to our Chef developers.
If I’m following the code correctly, kitchen/provisioner/chef_base gts all Mixlib::Install-y around here:
https://github.com/test-kitchen/test-kitchen/blob/6112653db9ff44cc7bdb5380203ae7095ba9e71f/lib/kitchen/provisioner/chef_base.rb#L506
Contrast this with chef-client-updater cookbook examples:
https://github.com/chef-cookbooks/chef_client_updater#examples
Whereby something like:
version ‘16’ => installs latest 16.x.y
version ‘12.13’ => installs latest 12.13.x
version ‘12.15.16’ => installs exactly this version
This also uses mixlib-install
https://github.com/chef-cookbooks/chef_client_updater#resources
So, tl;dr -- why can’t ktichen provisioner install strategies behave like chef-client-updater install strategies?
I have tried workarounds whereby a test cookbook would do the update using chef-client-updater, but the return code situation on windows is still not working as hoped (at least from what I had observed in my testing). And even IF this workaround did work, its yet-another workaround we have to train our devs on.
# concurrent kitchen verify
Using a cookbook with 4 or more suite as an easy case to reproduce, finish a successful ‘kitchen converge -c’ then try ‘kitchen verify -c’ the concurrent/parallel verify fails. Our current workaround is to run kitchen verify serially. This is yet-another workaround we have to train our devs on.
167 is probably the relevant one still open
https://github.com/inspec/kitchen-inspec/issues/167
https://github.com/test-kitchen/test-kitchen/issues/1568
https://github.com/test-kitchen/test-kitchen/issues/1307
https://github.com/inspec/kitchen-inspec/issues/119
c-rock says:
”
Its a bit tricky to implement parallel execution. The challenge is not so much in inspec itself, the challenge is that we use RSpec undercover and its creating a global state. I see two doable options:
instead of using the inspec’s ruby classes, we exec to the inspec bin with the arguments
we install inspec on each machine and (not preferred, since its way slower)
Essentially, we need to make sure that the execution context is not shared
Test Kitchen use is ubiquitous across Chef's user base. It clearly makes sense to address these issues. I'd consider this category of work "when" not "if".