Provisioning windows box with Chef-provisioning on azure from a mac

After spending about half a day trying to get vagrant-azure to work it became very clear, that as of this writing the driver is just not mature enough. It works pretty good for Ubuntu/Linux but the moment you try to provision windows boxes, it sets your laptop on fire.

Instead of wasting any more time on it, I decided to give v1 and v2 provisioning drivers a chance, followed by Test Kitchen. IIRC they all use different drivers, and while all are pretty solid at provisioning Linux boxes, support for WinRM is very spotty.

Authentication:

First challenge is to authenticate successfully via provisioning driver. While Vagrant accepts subscription id and path to .pem as parameters, provisioning needs a azureProfile.json.

To get that file generated, I installed azure-cli via brew `brew cask install azure-cli`

Next, import azure creds with `azure account import ../../Projects/Azure/myazure.publishsettings`
This command will generate the missing azureProfile.json in ~/.azure

Next, validate it works with `azure account list`

Chef-Provisioning piece:

Get a name of the box (ami) you'll be using: `azure vm image list | grep -i Win2012`

Next, hack up the simplest recipe that'll spin up a box:

`knife cookbook create azure_old`
content of recipe/default.rb:

require 'chef/provisioning/azure_driver'
with_driver 'azure'
machine_options = {
    :bootstrap_options => {
      :cloud_service_name => 'alexvinyar', #required
      :storage_account_name => 'alexvinyar', #required
      :vm_size => "Standard_D1", #required
      :location => 'West US', #required
      :tcp_endpoints => '80:80' #optional
    },
    :image_id => 'b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu-14_04_2-LTS-amd64-server-20150706-en-us-30GB', #required
    # :image_id => 'a699494373c04fc0bc8f2bb1389d6106__Windows-Server-2012-R2-20150916-en.us-127GB.vhd', #next step
    # Until SSH keys are supported (soon)
    :password => 'Not**RealPass' #required
}
machine 'toad' do
  machine_options machine_options
  ohai_hints 'azure' => { 'a22' => 'b33' }
end

Finally, run chef-zero (chef client in local mode): `chef-client -z -r azure_old`

If the above recipe fails, dont fail. Check the output, and see if it gets past the authentication piece. If it does, it's just a matter of getting chef-provisioning syntax correct.

Once the run finishes (Azure is slow) connect to the box with `ssh root@12.23.34.45` for centos or ubuntu@ip for ubuntu boxes.

Now the Windows piece

With the `azure vm image list | grep -i Win2012` command I got a list of boxes, and once the test run with ubuntu succeeds, I move on to Windows.

This is where I took a break and had a beer. But I published this post anyway because I'll finish it eventually.

Useful links:
http://azure.microsoft.com/en-us/documentation/articles/xplat-cli/
http://brew.sh/
https://unindented.org/articles/provision-azure-boxes-with-vagrant/

chef-base repo and workstation cookbook

A "chef-base" or "chef-repo" is a git repository which maps 1:1 to Chef organization hosted on the Chef server.  An organization in Chef server 12 is analogous to a single Chef server. Each of these "chef-base" Git repositories becomes the system of record for the global Chef objects (Environments, Roles, Data Bags) in a given organization.  This Git repository typically* does not contain cookbooks.

To setup chef-base a user should first create an empty git repository on VSO / GitHub / GitLab / etc..

It makes things slightly easier if none of the files are initialized, including readme and gitignore.

Next, user should execute "chef generate repo <name of github repo>" command. This will generate the skeleton for the repo.

The resulting skeleton folder should be pushed it its entirety to git repo.

Workstation cookbook

* One exception to not having cookbooks in chef-base is the workstation cookbook. 

The workstation cookbook is a shared cookbook for anyone using chef in an organization and provides a standardized way to work with chef. It also allows rapid on-boarding of new team members and ability to safely experiment with a new tools. 

It works well in Vagrant, but there is a major limitation, you can't run Test Kitchen inside a Vagrant. For best results, encourage teams to leverage internal or external cloud VM, where kitchen runs will create additional VMs in the same cloud.

A Vagrantfile can be placed in the root of the cookbook. This vagrant file has a couple of purposes:

• responsible for creating / destroying the workstation VM
• kicking off chef-client run
• easy access into the box via vagrant login
• mounting the local chef-base as a folder in a VM

.gitignore file should be modified to exclude all cookbooks with exception of the workstation cookbook.

Places to learn more:

https://github.com/mwrock/chef_workstation

https://github.com/scarolan/chef-dev-workstation

https://github.com/Nordstrom/chefdk_bootstrap 

<add yours here> or in the comments.

Random observations of a new publically facing Chef website.

First time using speakr.chef.co – musings and observations

I hope I won't hurt anyone's feelings by below, below is what I see as an engineer. Every time I see similar pages, I make a conscious choice to overlook these defects, it could be because I trust the site, or because I found the thing I need.

There is no way in hell I would know how to write an existing page, or actually implement the changes I noted. But what I find most fascinating about my job, is there is a guy somewhere in the company – every company - who knows exactly what comma to change to address the issue. If I were a business, I would seek these guys out, and reward them with titles, work from home schedules, “work on your own problem”, etc... It's just so un-economic and un-business like to lose them.

To business:

The experience has been an exercise in patience, but only due to an unfortunate coincidence of API incompatibility:

                The GeekWire even was announced using the Seattle address which excluded ZipCode:

                "Oct. 1-2, 2015, Sheraton Seattle, 1400 Sixth Ave."

                ( URL: http://www.geekwire.com/events/geekwire-summit-2015/ )

Executive Summary: Overall Conclusion:

This experience instantly demonstrated the inferiority of this form of entry, as compared to the auto context/syntax entry offered by modern companies. If this is an internally developed tool for anything other than a personal project, it should be replaced with a real tool meant for the job.

Error 1:

The speakr input fields request ZipCode as a mandatory field.

Result 1:

I had to visit google maps, enter the partial address to get the ZipCode to unblock myself.

Pretty sure my mom would get past this now.

Error2:

As @echohack says - Default matter. There is non-primary field that requests event start time. The defaults of the all 4 fields are set to 23:00. Meaning the entries are valid data type, but values for start date are totally off.

Musing:

I think an 8am is a nice default for "start time" on "start date".

Possible scenario: a study of booking data found that most people fly in a day before, and they actually do want the start time to be 11pm for previous day for a networked dinner.

After digesting things over, above doesn’t make sense, because this isn’t an expense system. An event system should specify actual start time.

Result 2:

Had to make a couple of extra clicks to change the start time.

Error 3:

On initial event creation webpage threw errors: "Invalid start date", "Invalid end date". Clicking on start/end date fields again and resubmitting the form resulted in successful creation message.

Result / Assumptions

The drop off rate here is probably very high. I actually almost gave up here.

I wonder if there is monitoring or metrics in place to see this kind of drop off. Unlikely, but I do wonder if there is an easy to implement “business flow” monitoring solution for that like Zabbix.

Personal research todo: I wonder if paid version of google analytics is significantly faster at page load times than free one.

Error 4:

Allowed creation of events which have already occurred.

Possible scenario:

Could be a feature too I guess.

Musings:

Might be a good idea to check if there is an anti-spam mechanism on event creation button.

Wonder if vanilla code coverage would pick something like this up, or if you need something like Fortify.

Error 5:

After successful event creation, that event would not show up in search results on events.chef.io.

Possible causes is the refresh job on events jobs is not triggered instantly, the page is not yet hooked up to events, past events are ignored as a result of a conscious choice (possibly even from business), or something else entirely.

Overall Conclusion:

This experience instantly demonstrated the inferiority of this form of entry, as compared to the auto context/syntax detection offered by modern companies. If this is an internally developed tool for anything other than a personal project, it should be replaced with a real tool meant for the job.