Creation of Air-gapped environment
Create DHCP option set
Create vpc 'alexv-manual automate in airgapped env'
set DNS resolution to Yes
set DNS Hostname to Yes
set DHCP option set to one above
Create a Windows 'jump box' inside VPC
network: vpc above
subnet - create new
VPC - vpc above
AZ - no pref
CIDR - same as vpc
refresh vpc field and select subnet
assign public IP - true
Network - default
Storage - default
(make sure you have enough free space to store all of the binaries needed inside VPC)
(i used 40 gigs) - this may mean that you have to expand default hard drive to occupy full HD space
Tag:
Name - alexv-jump box
SG:
create new SG 'jump box'
RDP - anywhere
HTTP - anywhere
HTTPS - anywhere
Select your keypair
** On your local Mac inside RPD app, enable folder redirection when you add this box.
set folder redirect to the location where your delivery.license file lives
on the windows box, install filezilla - to make it easy to transfer files
Installing Delivery
Create Chef Server
m3.medium
VPC - same as above
auto assign public ip - false
storage - change to 30
tag - alexv-chef-server
SG:
create new SG "chef server"
open port 22
open All ICMP
10`000-10`003
8989
HTTP
HTTPS
Keypair - select yours
Create Workflow server
click on chef-server, select more like this
VPC - same as above
Subnet - internal subnet
auto assign public ip - false
storage - change to 30
tag: alexv-Workflow-server
SG:
create new SG "Workflow server"
open port 22
open All ICMP
10`000-10`003
8989
HTTPS
HTTP
(maybe needed?) 9200 - due to elastic search get errors
(maybe needed?) 5672 - due to another elastic search failure?
Keypair - select yours
Create Windows (or *nux) build node
network: vpc above
subnet
VPC - vpc above
AZ - no pref
CIDR - same as vpc
refresh vpc field and select subnet
assign public IP - false
Network - default
Storage - default
Tag:
Name - alexv-windows build node
SG:
create new "windos build node"
open RDP - anywhere
open All ICMP
open 5984-5986 anywhere (for rdp)
Select your keypair
Internet Gateway:
create internet gateway - alexv-air gapped
attach to VPC (above)
Route:
when you create VPC, it created a route table
edit:
add 0.0.0.0/0 -> point at internet gateway
Save
Create 4 CentOS boxes to be environment nodes
medium size
HDD default
SG - copy from workflow server
Name SG "environment nodes"
create
Create 2 CentOS boxes to be build nodes
medium size
HDD - 15 gigs
SG - copy from workflow server
Name SG "build nodes"
create
Actually Install and Configure Automate
on the Chef Server and Automate node - follow directions===================
disable ipv6 in /etc/hosts
make sure they can ping each other
make sure they can resolve dns of each other
make sure they cant access internet
Jump Box (or workstation)
===================
copy target os binaries into jump box: chef server, automate, push jobs server, chefdk, chef manage, supermarket if needed.
copy binaries to correct server /tmp folder
copy chefdk for use on workstation as a management node
setup user ssh auth
ssh-keygen -t rsa -b 4096 -C "you@example.com"
Chef Server
===================
install chef server per directions
chef-server-ctl user-create alex alex alex@chef.io 'alexalex' --filename /tmp/alex_user.pem
chef-server-ctl org-create alex_org 'Fourth Coffee, Inc.' --association_user alex --filename /tmp/alex_org-validator.pem
install push jobs per directions:
sudo chef-server-ctl install opscode-push-jobs-server --path /tmp/opscode-push-jobs-server.x86_64.rpm
sudo chef-server-ctl user-create delivery delivery user deliver@chef.io 'alexalex' --filename /tmp/delivery_user_key.pem
sudo chef-server-ctl org-create automate_org 'org description' --filename /tmp/automate_org-validator.pem -a delivery
Install manage: (optional)
sudo chef-server-ctl install chef-manage --path /tmp/chef-manage-2.4.3-1.el6.x86_64.rpm
reconfigure chef, push, manage
on the Delivery server
===================
install delivery
setup command: sudo delivery-ctl setup \
--license /tmp/automate.license \
--fqdn ip-10-0-0-67.ec2.internal \
--key /tmp/chefserver/delivery_user_key.pem \
--server-url https://ip-10-0-0-80.ec2.internal/organizations/automate_org
copy all PEMs from chef server to delivery (validator, admin, delivery_user)
Enter name of your enterprise
example: alex_ent
(note: look for a bug here where enterprise is created, but admin creds are not displayed nor created in /etc/delivery/<enterprise-admin-credentials>)
(if bugged) creat enterprise manually
delivery-ctl create-enterprise alex_ent --ssh-pub-key-file=/etc/delivery/builder_key.pub
Copy ChefDk binary to /tmp/chefdk-0.18.30-1.el6.x86_64.rpm
install build node
sudo delivery-ctl install-build-node -I /tmp/chefdk-0.18.30-1.el6.x86_64.rpm -f 10.0.0.23 -u chef -P chef
Verify build node works with `knife node status`
this will query push jobs server for status of each node (different from knife status)
available means push jobs can communicate with the node (you will know that at least push jobs is running at this point)
Verify you can fire off a push job:
knife job start chef-client --search '*:*'
create user (via UI or CLI)
add public ssh key from workstations `ssh-keygen` step to the user
delivery ui -> user -> ssh pub key
Jump Box (or workstation)
===================
Install chefdk
configure knife.rb with delivery key for communication with chef server
example:
node_name 'delivery'
chef_server_url "https://ip-10-0-0-80.ec2.internal/organizations/automate_org"
client_key 'C:\Users\chef\.chef\delivery.pem'
trusted_certs_dir 'C:\Users\chef\.chef\trusted_certs'
# analytics_server_url 'https://cad-chef-server/organizations/cad'
cookbook_path 'C:\Users\chef\chef-demo\cookbooks'
fetch certs if needed
knife ssl fetch
verify knife works
knife node list
(or from delivery server)
knife node list -k /etc/delivery/delivery.pem -u delivery --server-url https://ip-10-0-0-80.ec2.internal/organizations/automate_org
Pull down all of the cookbook dependencies to be used in air-gapped env (i do it via berks)
mkdir repo
cd repo
chef generate cookbook staging (this will be the first test cookbook)
modify metadata.rb of seeding cookbook to include:
depends 'delivery-truck'
depends 'push-jobs'
depends 'build_cookbook'
depends 'delivery_build'
mkdir seeding
cd staging\.delivery\build_cookbook
run `berks vendor ..\..\..\seeding` to pull down all dependencies into a local folder
upload necessary cookbooks up to chef server
knife cookbook upload -o seeding -a
(or alternatively `knife cookbook upload delivery-truck --include-dep -o seeding`
test ssh auth to delivery box
ssh -l alex@alex_ent -p 8989 ip-10-0-0-67.ec2.internal
Configure delivery cmd - C:\Users\chef\cookbooks\staging\.delivery\cli.toml
in root of staging cookbook$ delivery setup -e alex_ent -o automate_org -s
ip-10-0-0-67.ec2.internal -u alex
make sure you can interact with delivery via delivery cli:
Verify API works
delivery api get users
delivery api get orgs
verify you can create a project
create a cookbook
`delivery init` inside that cookbook
First pipeline
===================
i'll use staging cookbook as it's a nice example
initialize delivery pipeline
inside staging cookbook run `delivery init`
bump metadata.rb if needed
modify config.json to exclude spec and test folders due to foodcritic testing them, leading to workflow epic failing on linting phase.
$ cat config.json
{
"version": "2",
"build_cookbook": {
"name": "build_cookbook",
"path": ".delivery/build_cookbook"
},
"delivery-truck":{
"lint": {
"foodcritic": {
"excludes" : ["spec","test"]
}
}
},
"skip_phases": [],
"build_nodes": {},
"dependencies": []
}
change Berksfile (of build cookbook)
Since you're not connected to internetz, you'll fail all phases of workflow due to Berksfile
change source to :chef_server
$ cat Berksfile
source :chef_server
# or your internal supermarket
metadata
group :delivery do
cookbook 'delivery_build'#, chef_api: :config
cookbook 'delivery-base'#, chef_api: :config
cookbook 'test', path: './test/fixtures/cookbooks/test'
end
#original
# group :delivery do
# cookbook 'delivery_build', git: 'https://github.com/chef-cookbooks/delivery_build'
# cookbook 'delivery-base', git: 'https://github.com/chef-cookbooks/delivery-base'
# cookbook 'test', path: './test/fixtures/cookbooks/test'
# end
add and commit changes
git add -u
git commit -m 'very descriptive comment'
delivery review
Bill of Materials:
===================
Filezilla (windows) - management node
Chef-server-core-12.9.1
delivery-core-0.5.346
push-jobs-1.1.6
chefdk-chefdk-0.18.30-1.el6.x86_64.rpm
note: seems like chefdk 17.17 doesnt work in isolated environment with a Yajl error
chefdk-18.30 for windows
chef manage rpm
supermarket rpm
berks vendor of `build cookbook`
should include all of the following:
build-essential
build_cookbook
chef-ingredient
chef-sugar
chef_handler
compat_resource
delivery-base
delivery-sugar
delivery-truck
delivery_build
dmg
git
mingw
packagecloud
push-jobs
runit
seven_zip
test
windows
yum
yum-epel
troubleshooting.
================
*) The setup command *may* create an enterprise for you. If you see that behavior, and do not get credentials as an output, you will have to delete the enterprise, and create it again using create-enterprise command.
*) node create command installs push jobs via this script:
https://github.com/chef/delivery/blob/114649cc8d6ddbf494a9666ef476e6a4b8523a7f/omnibus/files/ctl-commands/installer/gen_push_config.sh
..which is called by this script:
https://github.com/chef/delivery/blob/2ab9d4809e4ac1f237b52ee20088b1ac68d85af4/omnibus/files/ctl-commands/build-node/installer.rb#L217
Thanks for the write-up.. I was killing myself with the necessary cookbooks that need to be sneaker-netted over since the official docs only mention the two cookbooks.
ReplyDelete'berks vendor ..' is magical. Take a look at it.
DeleteA while ago I filed a bug so it no longer nukes existing folders in specified destination, so now it's much safes and much more versatile.