STIG, Windows2012, ACL, ACE, Registry, and aneurism

It's a very rewarding feeling, when after hours of research, google, and beating your head against the table, you finally come up with an answer. But I can't help feeling that 3 lines of code for 10 hours of research somehow diminish the sense of accomplishment. Not to say that I want 300 lines, but you know...

STIG and CIS benchmark documentation are as useful as they are impractical in the modern age. It's pages upon pages of useless manual steps. It's 2014.  TWENTY FOURTEEN!!  Unless you manage 4 systems that you NEVER rebuild, no one in their right mind is going to do this nonsense manually.

Example of nonsense:

Configure the policy value for Computer Configuration -> Windows Settings -> Security Settings -> Advanced Audit Policy Configuration -> System Audit Policies -> Global Object Access Auditing -> "File system" with the following: Select All Items.

The challenge at hand was automating implementation and validation of STIG V-1080:

Use the AuditPol tool to review the current configuration.Open a Command Prompt with elevated privileges ("Run as Administrator").  Enter "Auditpol /resourceSACL /type:File /view". ("File" in the /type parameter is case sensitive). The following results should be displayed:

Entry: 1
Resource Type: 
FileUser: 
EveryoneFlags:
FailureAccesses:
FILE_READ_DATA
FILE_WRITE_DATA
FILE_APPEND_DATA
FILE_READ_EA
FILE_WRITE_EA
FILE_EXECUTE
FILE_DELETE_CHILD
FILE_READ_ATTRIBUTES
FILE_WRITE_ATTRIBUTES
DELETEREAD_CONTROL
WRITE_DACWRITE_OWNER

The command was successfully executed.

Apparently it's not at all easy to apply Auditing rules to registry via CMD or Powershell.
Microsoft documentation leaves a lot to your imagination.

And as always, the answer was a mixture of Stackoverflow and blogs:

Step 1. Set the Audit rules manually.
Step 2. Get the SDDL from registry

$acl = get-acl hklm:\software
$acl.Sddl

Step 3. Apply SDDL via automation

$sddl = 'O:BAG:SYD:PAI(A;CI;KA;;;CO)(A;CI;KA;;;SY)(A;CI;KA;;;BA)(A;CI;KR;;;BU)(A;CI;KR;;;AC)S:AI(AU;CISA;KA;;;WD)'
$acl  = get-acl HKLM:\SOFTWARE
$acl.SetSecurityDescriptorSddlForm($sSDDL)
set-acl -Path HKLM:\\SOFTWARE -AclObject $acl

Step 4. Write Chef recipe and Serverspec integration test
Step 5. Realize that for some reason (either too much coffee or not enough) I confused two STIG rules and spent a mile walking in a direction of applying Auditing to HKLM:\Software instead of C:\ (Easy mistake to make I suppose)
Step 6. Change the script to actually apply auditing to all Drives instead.

Step 7. Make Chef recipe
Step 8. Make integration test via Serverspec

Useful links (in order of usefulness):

• http://www.leadfollowmove.com/archives/powershell/setting-filesystem-permissions-using-sddl-format/comment-page-1
• http://blogs.technet.com/b/ashleymcglone/archive/2011/08/29/powershell-sid-walker-texas-ranger-part-1.aspx
• http://stackoverflow.com/questions/13509667/how-to-get-audit-rule-in-acl-object-with-getauditrules-on-registry-key-in-powe

RHEL from a Windows guy

As ironic as it sounds, it wasnt until just a few months ago that I finally learned what RHEL stood for. That's 1994 - 2014. Nearly 20 years of blissful ignorance..

Now, I have a task to test a set of Chef recipes on RHEL 6.5 ... in Japanese.

So... questions are - 

1) How do I install RHEL?

2) How do I get it into Japanese?

... this is going to be as much of a brain dump, as a manual for myself to retrace my steps so that I can write a Chef cookbook automate everything when the time comes..

For the record - My workstation is a mac running OSX 10.9.4

1 - Getting RHEL box up.

Apparently you need some special license which you have to pay for. And I know nothing about it... nor does it have anything to do with what i'm trying to do. So to get some lift, I went with AWS.

Login, EC2 -> New server -> RHEL 6.5 -> Micro. Alternatively, you could spin up a CentOS box from http://www.vagrantbox.es/ which will be *almost* the exact same thing, but not really. So to avoid duplication I wanted actual RHEL. Also, you can use knife-ec2.

Even that has a tiny yak that one must shave.

  a) create an AWS key (aka AIM)

  b) save it locally

  c) fail to SSH using the key

  d) Google to find out that you need to change permissions on the key

      http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AccessingInstancesLinux.html

  e) CHMOD the pem key 

  f) ssh success

Ok. I'm in, now to validate if my RHEL is the actual version I need it to be.

   a) attempt a few obvious commands.

   b) fail

   c) google (http://serverfault.com/questions/89654/what-version-of-rhel-am-i-using)

   d) lsb_release -i -r
   e) above fails on some boxes, one alternative is to cat /etc/redhat-release

2 - How do I get it into Japanese

a) google...official documentation

b) read - https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/5/pdf/International_Language_Support_Guide/Red_Hat_Enterprise_Linux-5-International_Language_Support_Guide-en-US.pdf

Per kickstart docs, installation doesn't support Japanese / Chinese / few other languages, and defaults to English, which is great. One less test case to worry about and theoretically I won't have to deal with Kickstart since all configs can be done post OS.

On a related note, it sounds like there are 4 language settings:

* keyboard
* UI
* shell
* runtime  

btw - example kickstart files can be found here -
https://github.com/box-cutter/centos-vm/blob/master/http/ks6.cfg

c) fail.. system-config-language does not exist

..) google... http://linux-com.blogspot.com/2011/05/red-hatfedora-system-config-tools-guide.html

..) sudo yum install system-config-language

So.. i have no idea what the root password is, and I don't feel like resetting it or using EC2 UI again..

..) sudo su - 

..) system-config-language

..) scroll down..... tab tab.. ok!

..) Everything's still in English. now what? 

Turns out that by default, many linux distros will take on the locale of the system initiating the SSH connection. So there are a couple of other configuration steps needed to be done:

So in Summary:

• lsb_release -i -r   or    cat /etc/redhat-release
• sudo su -
• yum install system-config-keyboard
• yum install system-config-language
• system-config-keyboard (should obviously be done first, otherwise the language selection is in Japanese.)
• system-config-language
• modify "keytable" value in  keyboard setting file /etc/sysconfig/keyboard (set it to jp106)

Keyboard maps available maps here -- /lib/kbd/keymaps/i386/qwerty/jp106.map.gz

Good discussion on the topic:
Finally found something very useful...confirming my suspicions that changes were only temporary unless some file was actually modified to set them in stone:
http://www.howtoforge.com/changing-language-and-keyboard-layout-on-various-linux-distributions




-------------random unarranged stuff below this line -----------

3 - other stuff

bootstrapping AWS box is a little bit tricker now than it was a few months ago

gotta use the pem key, so you can literally grab the -i switch from SSH command and dump it into knife bootstrap

knife bootstrap 54.208.333.333 -x ec2-user -i ../../Projects/Amazon/alex.pem --sudo

Git aliases for my Powershell friends

For my windows friends.

In retrospect, I realized that OSX gives you some niftly little shortcuts that I pretty much take for granted. A list of Git Aliases that comes with Dotfiles is something I took for granted. It sure would be nice to have had these a year ago. Here it is for all of my Powershell Friends:

just ctrl+h alias to new-alias and dump it somewhere in your powershell's $profile

alias g=git
alias gCa='git add $(gCl)'
alias gCe='git mergetool $(gCl)'
alias gCl='git status | sed -n "s/^.*both [a-z]*ed: *//p"'
alias gCo='git checkout --ours --'
alias gCt='git checkout --theirs --'
alias gR='git remote'
alias gRa='git remote add'
alias gRb=git-hub-browse
alias gRl='git remote --verbose'
alias gRm='git remote rename'
alias gRp='git remote prune'
alias gRs='git remote show'
alias gRu='git remote update'
alias gRx='git remote rm'
alias gS='git submodule'
alias gSI='git submodule update --init --recursive'
alias gSa='git submodule add'
alias gSf='git submodule foreach'
alias gSi='git submodule init'
alias gSl='git submodule status'
alias gSm=git-submodule-move
alias gSs='git submodule sync'
alias gSu='git submodule foreach git pull origin master'
alias gSx=git-submodule-remove
alias ga='git add -A'
alias gam='git amend --reset-author'
alias gb='git b'
alias gbL='git branch -av'
alias gbM='git branch -M'
alias gbS='git show-branch -a'
alias gbX='git branch -D'
alias gbb='git bisect bad'
alias gbc='git checkout -b'
alias gbd='git b -D -w'
alias gbg='git bisect good'
alias gbi='git rebase --interactive'
alias gbl='git branch -v'
alias gbm='git branch -m'
alias gbs='git show-branch'
alias gbx='git branch -d'
alias gc='git commit --verbose'
alias gcF='git commit --verbose --amend'
alias gcO='git checkout --patch'
alias gcP='git cherry-pick --no-commit'
alias gcR='git reset "HEAD^"'
alias gca='git commit --verbose --all'
alias gcf='git commit --amend --reuse-message HEAD'
alias gci='git ci'
alias gcim='git ci -m'
alias gcl=git-commit-lost
alias gcln='git clean'
alias gclndf='git clean -df'
alias gclndfx='git clean -dfx'
alias gcm='git ci -m'
alias gco='git co'
alias gcp='git cp'
alias gcr='git revert'
alias gcs='git show'
alias gd='git diff'
alias gdc='git diff --cached -w'
alias gdi='git status --porcelain --short --ignored | sed -n "s/^!! //p"'
alias gdk='git ls-files --killed'
alias gdm='git ls-files --modified'
alias gdu='git ls-files --other --exclude-standard'
alias gdx='git ls-files --deleted'
alias gf='git fetch'
alias gfc='git clone'
alias gfch='git fetch'
alias gfm='git pull'
alias gfr='git pull --rebase'
alias gg='git grep'
alias ggL='git grep --files-without-matches'
alias ggi='git grep --ignore-case'
alias ggl='git grep --files-with-matches'
alias ggrc='git rebase --continue'
alias ggv='git grep --invert-match'
alias ggw='git grep --word-regexp'
alias gi='vim .gitignore'
alias giA='git add --patch'
alias giD='git diff --no-ext-diff --cached --word-diff'
alias giR='git reset --patch'
alias giX='git rm -rf --cached'
alias gia='git add'
alias gid='git diff --no-ext-diff --cached'
alias gir='git reset'
alias git='noglob git'
alias giu='git add --update'
alias gix='git rm -r --cached'
alias gl='git l'
alias glb='git log --topo-order --pretty=format:${_git_log_brief_format}'
alias glc='git shortlog --summary --numbered'
alias gld='git log --topo-order --stat --patch --full-diff --pretty=format:${_git_log_medium_format}'
alias glg='git l'
alias glo='git log --topo-order --pretty=format:${_git_log_oneline_format}'
alias glog='git l'
alias gls='git log --topo-order --stat --pretty=format:${_git_log_medium_format}'
alias gm='git merge'
alias gmC='git merge --no-commit'
alias gmF='git merge --no-ff'
alias gma='git merge --abort'
alias gms='git merge --squash'
alias gmt='git mergetool'
alias gnb='git nb'
alias gp='git push'
alias gpA='git push --all && git push --tags'
alias gpa='git push --all'
alias gpc='git push --set-upstream origin "$(git-branch-current 2> /dev/null)"'
alias gpf='git push --force'
alias gpl='git pull'
alias gplr='git pull --rebase'
alias gpp='git pull origin "$(git-branch-current 2> /dev/null)" && git push origin "$(git-branch-current 2> /dev/null)"'
alias gps='git push'
alias gpsh='git push'
alias gpt='git push --tags'
alias gr='git rebase'
alias gra='git rebase --abort'
alias grad='git remote add'
alias grc='git rebase --continue'
alias gri='git rebase --interactive'
alias grr='git remote rm'
alias grs='git reset'
alias grsh='git reset --hard'
alias grv='git remote -v'
alias gs='git status'
alias gsL=git-stash-dropped
alias gsS='git stash save --patch --no-keep-index'
alias gsX=git-stash-clear-interactive
alias gsa='git stash apply'
alias gsd='git stash show --patch --stat'
alias gsh='git show'
alias gshow='git show'
alias gshw='git show'
alias gsl='git stash list'
alias gsm='git submodule'
alias gsmi='git submodule init'
alias gsmu='git submodule update'
alias gsp='git stash pop'
alias gsr=git-stash-recover
alias gss='git stash save --include-untracked'
alias gst='git stash'
alias gstsh='git stash'
alias gsw='git stash save --include-untracked --keep-index'
alias gsx='git stash drop'
alias gt='git t'
alias gunc='git uncommit'
alias guns='git unstage'
alias gwC='git clean -f'
alias gwD='git diff --no-ext-diff --word-diff'
alias gwR='git reset --hard'
alias gwS='git status --ignore-submodules=${_git_status_ignore_submodules}'
alias gwX='git rm -rf'
alias gwc='git clean -n'
alias gwd='git diff --no-ext-diff'
alias gwr='git reset --soft'
alias gws='git status --ignore-submodules=${_git_status_ignore_submodules} --short'
alias gwx='git rm -r'
    && print .bundle       >>! .gitignore \
    && print vendor/assets >>! .gitignore \
    && print vendor/bundle >>! .gitignore \
    && print vendor/cache  >>! .gitignore'
alias spb='git checkout -b `sp | tail -2 | grep ''#'' | sed ''s/^ //'' | sed ''s/[^A-Za-z0-9 ]//g'' | sed ''s/ /-/g'' | cut -d- -f1,2,3,4,5`'

Learning GIT

Step 1. Use it
Step 2. Use it

Other resources to supplement step 1 and 2:

http://gitimmersion.com/
http://git-scm.com/book/en/Getting-Started
http://www.git-tower.com/learn/

How to do conditional include_recipe (how to use guards with include_recipe)

Lets say you want to include a recipe based on some state of the server. Obviously this is a server you inherited from some team, and rebuilding it is just not an option because server provisioning takes 2 weeks (or two months).

include_recipe "foo::bar" do 
  not_if {
  ::File.exists?("/etc/foo/bar")
  }
end


/\ That simple does't work, although it'd be awesome if it did (seems like a consistency thing to me). Reason for that is include_recipe is a method and not a resource. Which means it's evaluated at the compilation phase, and not at execution (which makes sense from building the resource collection perspective)


So, if you do want a conditional include_recipe, you want to do something like this instead:





some_variable =  {some logic to check for some state on a system} 

include_recide 'foo::bar' unless some_variable





Situation where this becomes super useful for example, is on windows, for chef runs which need to reboot a server and go on:



include_recipe 'cookbook::recipe' unless reboot_pending? 

Great link:

http://stackoverflow.com/questions/15961496/is-it-possible-to-chef-include-recipe-and-indicate-only-if-not-if-condition



The bug I filed on the issue and the resulting discussion:

https://tickets.opscode.com/browse/CHEF-5342

Toodles/

mini post - Chef (Opscode) - Simplifying Unit Testing

Saw something cool today in regards to unit testing.

Chefspec runs the first portion of your Chef run, the complication phase, but not the execution phase.

Next, it tests the resulting objects against the set of objects you defined in your unit tests. If they match, you pass. The key here is that you have all of the recipe stuff accessible to you from inside your unit tests, so you can use it to simplify your life. (where applicable, last thing you want is a set of tests that never fails)

For example, in situations where I build a string and some of the attributes are from upstream cookbooks - DNS server, Bill Gates favorite color, population of Ukraine...

So, with an attribute driven cookbook, a traditional unit test looks something like this:

Attribute:

default['download_dir'] = '/var/windows/rocks'

Recipe:

directory node['download_dir'] do
  owner "root"
  group "root"
  mode 0755
  action :create
  recursive true
end

Unit test (traditional):

it 'create directory' do
    expect(chef_run).to create_directory('/var/windows/rocks')
      .with_owner('root')
      .with_group('root')
      .with_mode(0755)
end

Unit test (simplified):

it 'create directory' do
    expect(chef_run).to create_directory(chef_run.node['download_dir'])
      .with_owner('root')
      .with_group('root')
      .with_mode(0755)
end

Now if you modify the attribute your unit test will still pass. Whether you want it or not is a whole different story, but so far it's been working much better than placing an exact string in the unit test.

mini post - Chef (Opscode) best practices series - Unit testing and TDD

The argument of TDD is great and all, but I do not think it's directly applicable to Chef.

Here is the idea - Chef offers a lot of resources, and a lot of ways to skin the cat/dog/lizard/etc.. So you simply can't write unit tests first, because you don't actually know which resources you're going to use to skin said animal.

What you CAN do, is write integration tests first. Because you DO know the ultimate outcome of your cookbook. And you can write Stubs for your unit tests ahead of time, if that makes you feel good.

Lets say you want to write a WSUS cookbook. The goal of the cookbook is to install WSUS role on your server. So your TDD (test driven development) for installing WSUS server would look something like this:

/> knife kitchen create wsus (or knife cookbook create wsus)

Your unit test would look very generic:

it 'installs some prerequisite that came with windows' do
end 

it 'downloads some prereq that didnt come with windows' do
end 

it 'installs the downloaded prereq' do
end 

etc...

Your integration tests on the other hand, would look very specific - Because you know ahead of time what constitutes a functional WSUS server.

 describe package('wsus') do
   it { should be_installed }
 end 

describe service('wsus') do
  it { should be_enabled   }
  it { should be_running   }
end 

describe port(3859) do
  it { should be_listening }
end

etc... 

IMHO, that's the best you can do when it comes to TDD with Chef. If you start with unit tests you're forcing yourself down the path of constantly re-writing your tests... over and over and over...

Here is an image of DNA relication.

Net user hidden switch

Holy moly... if only all of the internet problems were this easy to solve.

After creating and nuking 3 AWS boxes I finally broke down and regenerated the private cert. (I lost .pem file to interactively login a long time ago. Knife EC2 server create ftw!!)

Login and run my user_data script manually and get greeted with this 14 year old annoyance. Who in their right mind still runs win2k? Is it even supported?? Well, sadly we all know the answer to that question.

The password entered is longer than 14 characters.  Computers with Windows prior to Windows 2000 will not be able to use his account. Do you want to continue this operation? (Y/N) [Y]:

I nuked three boxes to find out my password was too long...

Combining last few drops of coffee with curiosity I ran the below command which to my complete and utter amazement succeeded. Half a google search later I find that this is a hidden switch in NET command... and now you know too

net user /add $user $password /yes

fyi: both /y and /yes work, and no matter how many times you type net help you wont get the answer you're looking for.

btw, here is the user data scrip:

<powershell>  #https://gist.github.com/vinyar/6735863;
  "[System Access]" | out-file c:\delete.cfg;  "PasswordComplexity = 0" | out-file c:\delete.cfg -append;  "[Version]"  | out-file c:\delete.cfg -append;  'signature="$CHICAGO$"'  | out-file c:\delete.cfg -append;    secedit /configure /db C:\Windows\security\new.sdb /cfg c:\delete.cfg /areas SECURITYPOLICY;    $user="UsernameGoesHere";  $password = "UserPasswordGoestHere";  net user /add $user $password;  net localgroup Administrators /add $user;    winrm quickconfig -q;  Enable-PSRemoting -force;  winrm set winrm/config/winrs '@{MaxMemoryPerShellMB="300"}';  winrm set winrm/config '@{MaxTimeoutms="1800000"}';  winrm set winrm/config/service '@{AllowUnencrypted="true"}';  winrm set winrm/config/service/auth '@{Basic="true"}';  winrm set winrm/config/service/auth '@{CredSSP="true"}';    netsh advfirewall firewall set rule name="Windows Remote Management (HTTP-In)" profile=public protocol=tcp localport=5985 new remoteip=any;</powershell>

Here is a random bit of awesomeness

Finding illegal characters somewhere in your chef cookbook / code

Use case:
I cant upload a cookbook / run powershell script / render web.config because something somewhere throws up on an illegal character.

One route to fix it, is to look at most critical files - web.config, json.config, README.MD and hope that the error is somewhere in there.

Well, that's one approach. About a year ago I would fire up notepad++ or BeyondCompare and look for iffy characters. It works, but it's stupid slow.

In case of Chef, the whole cookbook is processed, so theoretically the error could be anywhere. So, going through it one file at a time is just not practical. On top of that, I advocate automation, so... automation it is:

I've been learning OSX so I actually dont know the equivalent for windows, but since you dont have grep anyway, install the version that comes with -P and all is good.

brew install pcre
CD to the folder where you're breaking
pcregrep --color='auto'  -r -n "[^\x00-\x7F]" .
pcregrep --color='auto'  -r -n "[\x80-\xFF]" .

-r                              is for recursive
--exclude-dir .git      is to exclude .git from your search. You'll get a ton of hits here otherwise (not used in example above)
-n                             is to specify ascii range. Both work.

Source:
http://stackoverflow.com/questions/3001177/how-do-i-grep-for-all-non-ascii-characters-in-unix?s=4513f650-b50e-4611-8497-83fe400a3cc1

Way to test:  Grab a broken Repo and run the command to make sure you get expected broken file
Fix is here (https://github.com/stackforge/cookbook-openstack-common/commit/f4112fc77385a6044c2418f08da451aa57dfd247), so we're going to grab a pre fix commit

git clone https://github.com/stackforge/cookbook-openstack-common
git checkout 5e1a812a38c19a45b774862345f552c41836606f
pcregrep --color='auto'  -r -n "[\x80-\xFF]" .

Boom
Done

Post conference Rage blog - ChefConf

Having spent a week hanging out with the most genius people I've ever met, have really changed the way I look at user experience of the technology around us.

The third place goes to: Volvo
This example came from the CEO of the company during his keynote speech:
Volvo has a yearly GPS update package, which consists of 6 DVD disks, which have to be fed into the car over the course of 6 hours. AND!!! the car has to be running the whole time. Holy fucking shit, no way would a $60k+ car make you do that.

Sloppy seconds goes to: Hyatt
Preface: Hyatt has this pretty elevator system with dual factor authentication which doubles as first firewall in their customer defense system: after 7pm, you have to scan your card, AND the elevator takes you to your floor only. Neat, but mildly annoying, because card slots are small and very annoying to a drunk guy. Strike one.

That's minor though. The major flaw is that after the doors close, no new input can be added to the system. So, you're stuck riding in elevator till it stops.

And the first place goes to a Dealership:
Younker Nissan
3401 East Valley Rd
Renton, WA  98057

All of their emails come from: System Administrator <ileads_Younker_Nissan@webcrmmail.adpcrm.net>

... maybe not so much of user experience on that last one, but it definitely inspired this whole post. ...and now off to find a relevant XKCD post.

How about: What is Heartbleed in pictures

Automating WSUS 3.0 server with Powershell on Win 2k8r2

It was supposed to be easy... EASY!!!!

...this ended up being the first line that actually did anything relevant... obviously not fucking easy.

[reflection.assembly]::LoadWithPartialName("Microsoft.UpdateServices.Administration")

So this project started as usual, with a false hopes and unkept promises

PS C:\chef> Add-WindowsFeature OOB-WSUS

WARNING: Installation of 'Windows Server Update Services' is not supported on the command line. Skipping...

Oh Come on!!!!

So Step 1 - Install Pre - Reqs

"web-server", "Web-Asp-Net", "Web-Windows-Auth", "Web-Metabase", "File-Services" | foreach {if (!(Get-WindowsFeature $_)){add-windowsfeature $_}}

Step 2 - download / contribute to my Chef cookbook which will do it all for you

https://github.com/vinyar/wsus

The thing you probably really care about the most is here:

https://github.com/vinyar/wsus/blob/master/recipes/configure_wsus.rb 

Chef and Ruby and Gems and Windows

One of the many things in Chef ecosystem is Gems. Think of Gems as DLLs. They have versions, dependencies, and while MSFT figured this dependency hell years ago, Gems do not have that luxury.

One of the minuses is that when you install a gem, it pulls down depends, when you uninstall a gem, depends stay behind.

A - clean up unused gems

gem cleanup

B - nuke everything and start for scratch.

gem list | cut -d" " -f1 | xargs gem uninstall -aIx

C - read about bundler and track your depends in a gemfile. Each cookbook has its own gemfile. You always update your gemfile and never install gems manually

D - when your gems start complaining about missing dependencies see B

A good read:
http://shanky.org/2010/09/02/painlessly-remove-all-ruby-gems-on-windows/

Chef - Packer and generating images from custom ISO

If you want to develop quicker, you cant sit around waiting for your operations team to give you a server. .. and then give you another server when you destroyed it 20 minutes later. That's where you download VirtualBox, get Packer, Vagrant and go nuts.

The point of this post is to talk about packer templates you can get from GitHub. (templates have been moved into another project, but you can still get all of them for basic packer consumption from a dev branch of the same repo. Or just roll back master branch a couple of commits):

https://github.com/misheska/basebox-packer

If you want to use a ISO other than the one in a template, you'll need to specify a couple of parameters for packer, and get the SHA1 of the ISO you're using.

If you have OpenSSL, SHA1 is super easy: 

openssl sha1 c:\Images\raw\7601.17514.101119-1850_x64fre_server_eva
l_en-us-GRMSXEVAL_EN_DVD.iso

Alternatively, you can get this tool from the mothership:

http://www.microsoft.com/en-us/download/details.aspx?id=11533

Command:

packer build -only=virtualbox-iso -var 'iso_url=C:\Images\raw\7601.17514.101119-1850_x64fre_server_eval_en-us-GRMSXEVAL_EN_DVD.iso' -var 'iso_checksum=beed231a34e90e1dd9a04b3afabec31d62ce3889' win2008r2-enterprise.json

That's it. 5 minutes later you will have a brand spanking new virtual box for continuous destruction.

 ______________________________________ 
/ The best way to learn Chef is to use \
\ Chef. --Me                           /
 -------------------------------------- 
   \
    \
     \
       .--,--.          
       `.  ,.'        
        |___|        
        :o o:   O   
       _`~^~'_  |    
     /'   ^   `=)
   .'  _______ '~|
   `(<=|     |= /'
       |     |
       |_____|
 ~~~~~~ ===== ~~~~~~~~


 _________________________________________ 
/ Actually, my goal is to have a sandwich \
\ named after me.                         /
 ----------------------------------------- 
   \
    \
     \
       .--,--.          
       `.  ,.'        
        |___|        
        :o o:   O   
       _`~^~'_  |    
     /'   ^   `=)
   .'  _______ '~|
   `(<=|     |= /'
       |     |
       |_____|
 ~~~~~~ ===== ~~~~~~~~

Fixing AWS after upgrading to Powershell 4.0

I just got done scratching my head... After all, it's a PC, not an iPhone. I didnt think Amazon had a way to uninstall cmd-lets off my box remotely... or DID THEY!?

Well, no, they didnt. But apparently upgrading to powershell 4.0 does nuke all of the user modules, which sucks.

So. Step 1. Re-Install amazon modules (I just installed everything)

http://aws.amazon.com/powershell/  

Step 2.
Open new Powershell window and run:

Get-Module -ListAvailable

Step 3.

ImportSystemModules

Step 4. - Setup your private keys again. Oddly enough that part actually still works. So, hitting the ec2 commands works right after reinstalling cmdlets.

Set-AWSCredentials

(Get-EC2ImageByName WINDOWS_2008_BASE).imageid 

Step 5. If your creds were not carried over, or you never configured them, take a look at my earlier post

http://releaseengineer.blogspot.com/2013/09/managing-aws-with-powershell.html 

Ta-Da
You done

How to use same set of cookbooks with multiple Chef Servers

Simple answer is GIT. Well... probably the simple answer is to have multiple folders, each with corresponding .PEM files and configs, but then each knife.rb will have things like ../../cookbooks ../../.. and that's just too ghetto.

I assume by now you have GIT installed. If not, go install it - http://git-scm.com/
As a side benefit now you can do knife cookbook site install (instead of download)

Now you have a set of cookbooks - lets say your working directory is <somedir> and it has a few sub folders:

.chef
.kitchen (maybe, maybe not)
cookbooks
environments
roles
spec (maybe?, maybe not yet)
etc..

And you want to share them between two different Orgs - lets say, getchef.com chef server, and a private Server behind 13 firewalls and 14 VPNs..

Easy. Add your configs under source control, and setup two branches. One for each Chef server you'll be dealing with. Then you'll just check out the branch corresponding to the server config you want to use.

* Test that your current server works (knife cookbook show or knife client list)
* Go to .chef folder, or wherever your knife.rb and various settings live.

git init (initialize bare git repo)
git add . (add all files to repo)
git commit -m 'bla' (commit files to git repo)

You just placed your configs under version control in 'master' branch. YAY (*think of Dallas cheer leaders*)

Now, we will clone the master branch and configure .PEM keys and URL for the other server

git checkout -b local master (you just cloned master into local - good idea to have this as a backup of master)git checkout -b russia master (you just cloned master into russia)git checkout russia (switch to working on a specific branch)Add new .PEM files (your validator/pem may have different names.)Make changes to knife.rb (URL and if needed, file names. The rest should stay the same)Connect to the VPN's and/or whatever else. Test the new connections. (knife cookbook show)git add .git commit -m 'bbbla'

now any time you want to change which server you're working with just do git checkout local or git checkout russia. 

Boom
Done

Easy way to benchmark ruby code

As I was pounding away at the language I barely understand, I started thinking... which of the two approaches is faster:

A:    Let RegEx do a case insensitive search or
B:    Downcase all of the output, and let the regex run.

`auditpol /get /subcategory:\"#{@category}\"`.match(/#{test}/i)
or
`auditpol /get /subcategory:\"#{@category}\"`.downcase.match(/#{test}/)

Well, apparently there is an incredibly .. retardedly easy way to satisfy that curiosity:

 The simplest way to measure your Ruby code is with Benchmark.measure

require 'benchmark'
require 'bigdecimal/math'

# calculate pi to 10k digits
puts Benchmark.measure { BigMath.PI(10_000) }

More here: http://rubylearning.com/blog/2013/06/19/how-do-i-benchmark-ruby-code/

Just wow.... I don't think I ever had it that easy with measure object in powershell... to be perfectly honest, that object in powershell is borderline useless.

Managing AWS with powershell - part 3

Pure Magic...

How to get Information for all of the nodes you have with passwords and FQDS... Like this:

(Get-EC2Instance |? {$_.RunningInstance.keyname -like '<YOUR KEY NAME>'}).RunningInstance | select instanceid, PublicDnsName, IpAddress, @{Label="password";Expression={Get-EC2PasswordData $_.instanceid -PemFile '<Path to your .PEM file...C:\Bla\bla.pem>'}}, @{label="test";expression={(Get-EC2Instance $_.instanceid).RunningInstance.Tag.value}} | ft -autosize

 

Enjoy

Idempotence in Chef on Windows via WMI / Powershell - Light at at the end of the tunel

As part of writing Chef recipes, the big thing is to run code once and only once.

Well, Chef is Ruby, running on top of windows. To make things more interesting, the native wmi providers come in the form of a community GEMs.

Example:

only_if { WMI::Win32_Service.find(:first, :conditions => {:name => 'chef-client'}).nil? }

The best way to find / test WMI functionality is with Powershell... Well..Here are a few things that make life just a little easier:


1: This bad boy automatically generates WMI query in powershell. Talk about free.

http://technet.microsoft.com/en-us/library/ff730935.aspx
Download here: http://www.microsoft.com/en-us/download/details.aspx?id=24121

2: Which namespace should you use???

Powershell 3 (Server 2008+) gives you the answer for free!!!!
Example1: Get-CimClass -MethodName *bios*
Example2: Get-CimClass -ClassName *bios*

What do you get?

    NameSpace: ROOT/cimv2

CimClassName                        CimClassMethods      CimClassProperties
------------                                  ---------------               ------------------
Win32_SMBIOSMemory       {SetPowerState, R...   {Caption, Descripti
CIM_BIOSElement                 {}                                {Caption, Descripti
Win32_BIOS                           {}                                {Caption, Descripti
CIM_VideoBIOSElement       {}                                {Caption, Descripti

You get a free Powershell Query:

 

Holy Crap!! That was easy.

Managing AWS with powershell - part2

Passing in User Data to server create command

Interestingly enough, you have to pass userdata to new instance in a specific format. It has to start with <powershell> and end with </powershell>

..but then in gets more interesting.

You have to pass in your custom script in the Base64 format.

Converting your file to Base64

$string64 = gc .\aws-userdata.ps1 -Encoding byte
$string64_string = [System.Convert]::ToBase64String($string64)
$string64_string

Creating node with user data

New-EC2Instance -ImageId ami-monkey -MinCount 1 -MaxCount 1 -keyname monkey -InstanceType m1.medium -Verbose -UserData $string64_string 

 

Managing AWS with powershell - part 1

So, as ironic as it may be, EC2 is significantly faster and easier to use than Azure. Like.. 2 minutes vs 15 per server faster. Not only that, to get Azure powershell running, you need to download 200 megs of crap, including SQL server lite, hello kitty theme, and aol 2.5. Amazon on the other hand is a 5 meg download which includes a couple of powershell cmdlets for managing all of Amazon... it does have a few caveats.

Get your creds automated

Set-AWSCredentials 

note: http://docs.aws.amazon.com/powershell/latest/userguide/pstools-appendix-signup.html 

note: http://docs.aws.amazon.com/powershell/latest/userguide/specifying-your-aws-credentials.html 

How to get AMI name for creating a server 

(Get-EC2ImageByName windows_2008_base).imageid

How to create a server:

New-EC2Instance -imageid <ami-monkeys> -mincount 1 -maxcount 1 -keyname monkeys -instancetype m1.medium

How to get connection information for the box (or 20 boxes) you've just cooked up 

$a = Get-EC2Instance

$b = $a | ?{$_.reservationid -like "whatever is the reservation you got from new-EC2Instance"}

$b.RunningInstance | select PublicDnsName, ipaddress, privateipaddres

How to get password for the node you just made?

Get-EC2PasswordData -InstanceId $c.RunningInstance.instanceid -PemFile 'C:\Users\monkeys\monkey.pem'
or in case of multiple boxes just make a loop

With all of the above said, and as awesome as AWS is, there is a horrible horrible default which makes absolutely no sense:

help New-EC2Instance -full

-MinCount <System.Decimal?>

Minimum number of instances to launch.  If the value is more than Amazon EC2 can launch,  no instances are launched at all.

         Constraints: Between 1 and the maximum number  allowed for your account (default: 20).

-MaxCount <System.Decimal?>

Maximum number of instances to launch.  If the value is more than Amazon EC2 can launch, the largest possible  number above minCount will be launched instead.

         Constraints:  Between 1 and the maximum number allowed for your account  (default: 20).