Easy way to benchmark ruby code

As I was pounding away at the language I barely understand, I started thinking... which of the two approaches is faster:

A:    Let RegEx do a case insensitive search or
B:    Downcase all of the output, and let the regex run.

`auditpol /get /subcategory:\"#{@category}\"`.match(/#{test}/i)
or
`auditpol /get /subcategory:\"#{@category}\"`.downcase.match(/#{test}/)

Well, apparently there is an incredibly .. retardedly easy way to satisfy that curiosity:

 The simplest way to measure your Ruby code is with Benchmark.measure

require 'benchmark'
require 'bigdecimal/math'

# calculate pi to 10k digits
puts Benchmark.measure { BigMath.PI(10_000) }

More here: http://rubylearning.com/blog/2013/06/19/how-do-i-benchmark-ruby-code/

Just wow.... I don't think I ever had it that easy with measure object in powershell... to be perfectly honest, that object in powershell is borderline useless.

Managing AWS with powershell - part 3

Pure Magic...

How to get Information for all of the nodes you have with passwords and FQDS... Like this:

(Get-EC2Instance |? {$_.RunningInstance.keyname -like '<YOUR KEY NAME>'}).RunningInstance | select instanceid, PublicDnsName, IpAddress, @{Label="password";Expression={Get-EC2PasswordData $_.instanceid -PemFile '<Path to your .PEM file...C:\Bla\bla.pem>'}}, @{label="test";expression={(Get-EC2Instance $_.instanceid).RunningInstance.Tag.value}} | ft -autosize

 

Enjoy

Idempotence in Chef on Windows via WMI / Powershell - Light at at the end of the tunel

As part of writing Chef recipes, the big thing is to run code once and only once.

Well, Chef is Ruby, running on top of windows. To make things more interesting, the native wmi providers come in the form of a community GEMs.

Example:

only_if { WMI::Win32_Service.find(:first, :conditions => {:name => 'chef-client'}).nil? }

The best way to find / test WMI functionality is with Powershell... Well..Here are a few things that make life just a little easier:


1: This bad boy automatically generates WMI query in powershell. Talk about free.

http://technet.microsoft.com/en-us/library/ff730935.aspx
Download here: http://www.microsoft.com/en-us/download/details.aspx?id=24121

2: Which namespace should you use???

Powershell 3 (Server 2008+) gives you the answer for free!!!!
Example1: Get-CimClass -MethodName *bios*
Example2: Get-CimClass -ClassName *bios*

What do you get?

    NameSpace: ROOT/cimv2

CimClassName                        CimClassMethods      CimClassProperties
------------                                  ---------------               ------------------
Win32_SMBIOSMemory       {SetPowerState, R...   {Caption, Descripti
CIM_BIOSElement                 {}                                {Caption, Descripti
Win32_BIOS                           {}                                {Caption, Descripti
CIM_VideoBIOSElement       {}                                {Caption, Descripti

You get a free Powershell Query:

 

Holy Crap!! That was easy.

Managing AWS with powershell - part2

Passing in User Data to server create command

Interestingly enough, you have to pass userdata to new instance in a specific format. It has to start with <powershell> and end with </powershell>

..but then in gets more interesting.

You have to pass in your custom script in the Base64 format.

Converting your file to Base64

$string64 = gc .\aws-userdata.ps1 -Encoding byte
$string64_string = [System.Convert]::ToBase64String($string64)
$string64_string

Creating node with user data

New-EC2Instance -ImageId ami-monkey -MinCount 1 -MaxCount 1 -keyname monkey -InstanceType m1.medium -Verbose -UserData $string64_string 

 

Managing AWS with powershell - part 1

So, as ironic as it may be, EC2 is significantly faster and easier to use than Azure. Like.. 2 minutes vs 15 per server faster. Not only that, to get Azure powershell running, you need to download 200 megs of crap, including SQL server lite, hello kitty theme, and aol 2.5. Amazon on the other hand is a 5 meg download which includes a couple of powershell cmdlets for managing all of Amazon... it does have a few caveats.

Get your creds automated

Set-AWSCredentials 

note: http://docs.aws.amazon.com/powershell/latest/userguide/pstools-appendix-signup.html 

note: http://docs.aws.amazon.com/powershell/latest/userguide/specifying-your-aws-credentials.html 

How to get AMI name for creating a server 

(Get-EC2ImageByName windows_2008_base).imageid

How to create a server:

New-EC2Instance -imageid <ami-monkeys> -mincount 1 -maxcount 1 -keyname monkeys -instancetype m1.medium

How to get connection information for the box (or 20 boxes) you've just cooked up 

$a = Get-EC2Instance

$b = $a | ?{$_.reservationid -like "whatever is the reservation you got from new-EC2Instance"}

$b.RunningInstance | select PublicDnsName, ipaddress, privateipaddres

How to get password for the node you just made?

Get-EC2PasswordData -InstanceId $c.RunningInstance.instanceid -PemFile 'C:\Users\monkeys\monkey.pem'
or in case of multiple boxes just make a loop

With all of the above said, and as awesome as AWS is, there is a horrible horrible default which makes absolutely no sense:

help New-EC2Instance -full

-MinCount <System.Decimal?>

Minimum number of instances to launch.  If the value is more than Amazon EC2 can launch,  no instances are launched at all.

         Constraints: Between 1 and the maximum number  allowed for your account (default: 20).

-MaxCount <System.Decimal?>

Maximum number of instances to launch.  If the value is more than Amazon EC2 can launch, the largest possible  number above minCount will be launched instead.

         Constraints:  Between 1 and the maximum number allowed for your account  (default: 20).

Powershell - converting array output into string (OpenSSL ...)

In my specific example, I was trying to get OpenSSL output to show up as a string in the least amount of typing (this is for a class I'll be teaching), meaning each student would have to read the slide, and type what's on it. Meaning that each extra bracket of code students type, could result in some unpredictable disaster which would stop the whole class from moving forward. So, I wanted to come up with the most elegant way possible.


Example:

PS\> openssl rand -base64 512

vq6liIBbS0vy5XXUQiknc8Mw/Bzm5YwjYJKkv34wns0w7OGXcXpXN4mutLmlcJ7h
N7P8557UtZW2JPwhrh6KbrtqK6SzvQK0uZvAejt074gOS0yWkVvnQC6/y4GWpfkA
l93McZpiHB7dAUxglHWPdA2u3+wYfJMt4VkCYAFTqyDyFyRxhi9KWhRdE55OyZQz
P92tVM8Vjo7rQVroDdW/SvQTPklAzJcwbubuBCfXvENWwhcXFEOyLhjftoxSEojR
qTraYr5J6CfMs9ELedRKjEUgHeXcnarBf0BbjliDwzeVQ7DcFGeq6i4aRed4U+1X
OuH5ewOI2qqrY3sQtWnG52X2BYWhi8Bs0G6h1SVvn7LsCLO3HGjB/cyLcujYQpcv
HMw8EXXndOvSXMif/N66RjRbafn0ZNjlCB1KtdNIy8y7gPJCkf6UbJuypZGWHlnd
wktJV1L4S7kKNKyaDX+9tZoTiZhN50qTYFZ+dQ9E5ItIBxoHc5UuQvK334nm8y70
Czpq1lfj1S9CSEvkcCaMkOnZOx+mb1rPcIm4fg6pA+F10m1t4Eph4m3QLxY7H4RE
yfxsZhWbE+I/epsODOeVi3VzHzz3UuNKuA+RvLD2U8QGDgyl9DRIgKGmMaShN7oa
6+OWoKeX4spSCkFIPwuPlcLAkVSBDEIrwlAIi+9hQr8=

But what you really want is a string... The reason is, in Chef the data_bag object can be encrypted. So user names, password, data base keys are all safe and sound behind OpenSSL (pun retroactively intended). However, decrypt takes a string, and Powershell generates an array. So, I needed to convert array to string in powershell in the most elegant way possible.

I am not* going to bore you with the many failed attempts, so here is the best answer I came up with*:

PS\> -join $(openssl rand -base64 512)

Yup. That's it.

A really neat side effect, is that in every class I have a few people tell me this "join" is awesome and they had no idea it was possible.  This is why I dig what I do now.

* I didn't actually come up with the -join by itself, this dude here did: http://stackoverflow.com/questions/7723584/powershell-how-to-convert-array-object-to-string

I lied about not boring you... here are the various semi-failed attempts:

•     [string]$(openssl rand -base64 512) -replace " ",""
•     openssl rand -base64 512 | %{write-host $_ -NoNewline}
•     Write-Host -Object $(openssl rand -base64 512) -Separator ""
•     openssl rand -base64 512 | %{[string]$a+=$_}
•     $result = $result -replace "`t|`n|`r",""
•     ...this one is cool, but makes changes to $ofs:
            $ofs="";[string]$(openssl rand -base64 512)

Pushd / Popd / Powershell - let our powers combine

Simple, redefine your CD command as PUSHD

... I honestly cant believe that it just hit me now, after years of using powershell... this is just brilliant. I am a genius*!!!

remove-Item alias:cd
new-alias cd pushd

..and now any just fire off POPD any time you want to go back.
...and now that you're absolutely in love with this, you will never go back, so go ahead and make it a part of your profile

"remove-Item alias:cd;new-alias cd pushd" |out-file $profile -append

MAGIC!!!

unrelated image:

*pending scientific study

Chef / Ruby... What editor to use?

Well... I've been living in the powershell world for the last few years, and writing Chef recipes (Ruby) has blown the lid off my teapot.. the number of brackets and commas, is borderline absurd.

Sublime to the rescue (it's totally free - minus one annoying occasional popup)
http://www.sublimetext.com/

Single best feature of the tool is bracket / comma / braces / etc.. auto completion.

I've switched to Sublime exclusively for Ruby, but still using Notepad++ for Powershell... however! Today I was referred to two excellent plugins which look VERY promising:

https://github.com/n1k0/SublimeHighlight (Export highlighted code into RTF or HTML - great for Demos)

and https://github.com/SublimeText/PowerShell (Syntax highlighting for powershell in Sublime)

A simple installation doc:

http://dilutedthoughts.com/dilutedthoughts/2012/12/06/adding-powershell-package-in-windows-for-sublime-text-2

Completely unrelated image of the thing I'm building..Sous Vide Heating Immersion Circulator

http://seattlefoodgeek.com/2010/02/diy-sous-vide-heating-immersion-circulator-for-about-75/

Three hints for Active Directory

1: How to get distinguished name through powershell from a computer without using Active Directory module

$filter = "(&(objectCategory=computer)(objectClass=computer)(cn=$env:COMPUTERNAME))"
([adsisearcher]$filter).FindOne().Properties.distinguishedname

Compliments to Shay: http://stackoverflow.com/questions/11146264/get-current-computers-distinguished-name-in-powershell-without-using-the-active

2: How to get Domain controller to see Security Policies.

Apparently, as part of the design, AD will not see security properties from it's own GPO unless they are defined at the global scope.

Which mean, if you're trying to lock down AD to some security spec, such as CIS, you'll have to define hardened policies at the global scope. If you dont want your nodes to be as locked down as the AD, you'll have to enforce the GPO with relaxed settings at the node OU to overwrite global.

A bit of a headache if you ask me.... especially since it took me nearly a day and a half to find an answer.
http://support.microsoft.com/kb/259576

3:  Exposing MSS settings (some hardening specs such as CIS calls for it)

Note: You can edit GPO from any server where GPMC is installed, so this tool can be installed on any 2k8 server in the domain (no reason to clutter up the AD).

* Install Security Compliance Management Toolkit.
(http://www.microsoft.com/downloads/details.aspx?FamilyID=5534bee1-3cad-4bf0-b92b-a8e545573a3e&displaylang=en)
-- the installer will probably fail to work...it's probably by design.

* Install SQL express manually. Use any version.
* Re-run Security Compliance Management Toolkit installer.
* Click the Start, click All Programs, Microsoft Security Compliance Manager, Local GPO.
* Install LocalGPO.MSI
* Go to the folder where you installed the MSI (default is C:\Program Files (x86)\LocalGPO) and
* From Administrative command prompt run: cscript LocalGPO.wsf /ConfigSCE and then press ENTER.
* Open GPO editor
* Navigate to Computer Config -> Policies -> Windows settings -> Security Settings -> Local Policies -> Security Options

BOOM MSS is now visible.

Subduing sublime - moving out of quotes with ease

Yet another entry shamelessly stolen from another blog.

I just started using Sublime and I really dig the bracket and quote auto completion, but it immediately became apparent to me, that it pisses me off as much as it helps me. There is no really good way to move out of the brackets without either typing them out - which defeats the purpose of auto completion in the first place, using arrow keys, using end key, or god forbid using the mouse.

The only built in functionality is control+enter which gets you a new line - not something you want every time.

The answer is to create your own key binding - below example gets you out of all kinds of brackets (the kind I use at least) and gives you a good template for setting it up for your own use.

For practice I setup 2 keys to do same thing: Tab and Enter. Might end up changing it later, or making it Shift+Enter or something


Preferences > Key Bindings -- User



{ "keys": ["enter"], "command": "move", "args": {"by": "characters", "forward": true},
  "context":
  [
      { "key": "following_text", "operator": "regex_contains", "operand": "^[})\\]'\"]", "match_all": true },
      { "key": "preceding_text", "operator": "regex_contains", "operand": "[{(['\"]", "match_all": true },
      { "key": "auto_complete_visible", "operator": "equal", "operand": false }
  ]
},
{ "keys": ["tab"], "command": "move", "args": {"by": "characters", "forward": true},
  "context":
  [
      { "key": "following_text", "operator": "regex_contains", "operand": "^[})\\]'\"]", "match_all": true },
      { "key": "preceding_text", "operator": "regex_contains", "operand": "[{(['\"]", "match_all": true },
      { "key": "auto_complete_visible", "operator": "equal", "operand": false }
  ]
}


Plug goes out to: http://www.codejury.com/fixing-some-of-sublime-texts-annoyances/

Hidden escape character in SETX

Shamelessly stolen but very useful

I recently discovered that set and setx behave completely different when it comes to escape characters.

Example:
set bob="C:\Program Files (x86)\Notepad++\notepad++.exe"
setx bobx "C:\Program Files (x86)\Notepad++\notepad++.exe"

right? Seems correct, but one will work, while the other wont.
Executing %bob% will launch notepad while %bobx% will error out.

why? Well, apparently
setx bobx "C:\Program Files (x86)\Notepad++\notepad++.exe" equates to using set with no quotes - like so:
set bob=C:\Program Files (x86)\Notepad++\notepad++.exe

What we find, is that setx has it's own wonky magic, and if we wanted to escape a character, we use  \ ... which doesn't come up in documentation. Awesome, yea?

setx bobx "\"C:\Program Files (x86)\Notepad++\notepad++.exe"\" which would equate to 
..set bob="C:\Program Files (x86)\Notepad++\notepad++.exe"

now %bobx% will start notepad++



free plug for one of the 10^10 blogs that caught it - http://batcheero.blogspot.com/2008/03/setx-escape-character.html

Downloading SVN without SVN client (aka - how to get wordpress themes)

My friend asked me to migrate her Wordpress.com blog to a private Wordpress.org site.

Wordpress.org setup takes 5 minutes and is not worth writing about. However after migration, the theme was missing.

Apparently, the themes do not migrate, but they are available via SVN here:
https://wpcom-themes.svn.automattic.com/ideation-and-intent (and others)

But... I do NOT want to install SVN, and I could get them with WGET.
off topic, but neat write up on wget here: http://www.linuxjournal.com/content/downloading-entire-web-site-wget

But I did find a way to download any SVN repository as a ZIP file w.o any installs :

http://svndownload.eu5.org/

BOOM!!

Netscalers are no laughing matter

Found this conversation I had with an Expedia* network engineer about a year back. Made me shed a tear of laughter and sadness ...

-----

2:54 PM

once upon a time a boss was told that he was responsible for anything bad that happens on all 130 netscalers at expedia

said boss then immediately changed all passwords on all netscalers and asked his network team to manage these appliances

said network team had no clue how to manage said appliances so the tickets backed up

and up

and as the tickeets backed up...

network engineers got fired for messing up the netscalers

and then so did the boss

until there was no one left to manage the appliances

so then .. the natives got restless

.. as teams started to build vips and have issues with vips... the network team was told to fix vips and all vip issues

but buried under piles of tickets the network team still wasn't able to fix the vips

because they couoldn't reach their keyboards

however, it was said by the powers that be that teams would then once again be able to build vips in the realm as long as the network team peer review all changes

yesh, and eli and them can pretty much just ping the brave knight Tajon to peer them and all is well

while the poor, poor (literally broke) network team continue to flail about under piles of tickets

the end

*He was so proud of having worked there, it's not even on his linked in profile. 

Short and sweet!!

Shortest post ever - lots of good info here.

http://www.theregister.co.uk/2013/07/15/powershell_third_party_guis/

Automation Feedback and making it useful.

We've been running tons of automation. Automation to deploy to a server. Automation to clean the logs. Automation to check the status.. and even automation to check on other automation.

But how do you keep your email box from exploding with status email or just noise?

1: Ask yourself - do you need to know if things are healthy?
2: Can you have self-healing mechanisms in place if things are broken.
3: Does your company have a corporate scheduling engine (Bladelogic, ControlM, or even shared SQL Jobs box)?


Here are a few tips that I found useful:

• Sending emails on failure only
• Including reasonable self-healing steps into every script or carving them out into a validation script.
• for example: Auto cleanup for out-of-space issues, restart IIS, kill off handles to locked file, restarting a server, restart a node in AppFabric cluster, etc..
• Use TEE command -  pipe console output of your scripts to a log. Any errors not explicitly handled by your scripts will end up at the council and may end up being completely missed.
• Logs naming convention should allow you to easily group and sort files.
• If you're using scheduled tasks, have a separate job checking the status of things. It should run under a different account, with a different password expiration date.
• Keep the email body short.
• Include the actual error into the email
• Include the UNC (\\servername\logs\server_name\log_name.html) into the body of the email.
• Include UNC to the TEE command.
• Keep the Subject of your emails Informative and Generic:
  "%Servername% failed" is borderline useless as far as subject lines go.
• You can put all of your logs into a folder accessible through IIS (or tomcat) and just enable directory browsing.

These are just a few ideas.. but you get an idea.

my favorite comic this week..

How to ping with unplugged network cable?

Sample scenario:

You've rebooted a router and you want to know when the network connectivity is restored.

Typical PING -t  will not work (test-connection might, but I have not play with it a whole lot)

C:\Users\bob>ping -t www.google.com
Ping request could not find host www.google.com. Please check the name and try again.

So... we do a simple alternative by wrapping ping into a loop:

C:\Users\bob>powershell
PS C:\Users\bob> for (1..1000){ping www.google.com;start-sleep 10}
Ping request could not find host www.google.com. Please check the name and try again.
Ping request could not find host www.google.com. Please check the name and try again.
Ping request could not find host www.google.com. Please check the name and try again.
Ping request could not find host www.google.com. Please check the name and try again.
Pinging www.google.com [74.125.129.99] with 32 bytes of data:
Reply from 74.125.129.99: bytes=32 time=20ms TTL=48
Reply from 74.125.129.99: bytes=32 time=20ms TTL=48
etc..


Now, if you want to get fancy, you can use while loop and add ($?) or (!$?) and pair it up with Send-MailMessage or other fanciness of your choice.

GG

What does release engineer do?

Preface: I work at Expedia.

I went to a graduation reunion yesterday, and we were supposed to give advice to new graduates. Since most people don’t know what Release Engineering is, or what anyone besides a programmer does, I came up with a great way to explain what I do (or so I thought).

Xxx: are you a programmer?
Me: no, I’m a release engineer
Xxx: what’s a release engineer?
me: Do you know what a wedding planner is?
xxx: yes.
me: Ok. Imagine a couple getting married. First they get engaged, and then a bunch of shit has to happen, and they have a big party. So that's what a wedding planner does.
xxx: yea totally
me: In IT world, there is a programmer and a customer. Programmer makes something, then a bunch of shit has to happen, then a customer sees it. I do all the shit in a middle
xxx: I didn’t know Expedia does weddings.

me: ......... I need a beer.