Thursday, March 28, 2013

Basic installation of Chef using a managed server in 10 minutes

Chef is quite a comprehensive tool, capable of deploying configurations, code and cloud elements. We'll just do some basic steps, for more information please do visit their website.

As a first step, create yourself a free account at their website. Upon first login, go to your Organization and download the generated keys and knife configuration:


You will download a file called knife.rb and a certificate named <company>-validator.pem. For us company will be andreu2.

Now let's get your private certificate (node certificate). Click on your username, then View Profile and then get private key:


Now,  install the client on your desktop. The website recommends doing it this way:


$ curl -L https://www.opscode.com/chef/install.sh | sudo bash
 
It didn't work on my Debian 7, so I installed the packages from Debian Repository:
 
 $ sudo apt-get install rubygems chef

(it will install quite some packages)

Any workstation interacting with Cheff need to have the chef-repo. Create the folder ~Development and get the git repository:

$ mkdir ~/Development
$ git clone git://github.com/opscode/chef-repo ~/Development/chef-repo

Now we have a sub folder chef-repo inside Development. Create a sub folder inside named .chef and copy the knife.rb and both certificates:

$ mkdir ~/Development/chef-repo/.chef
$ cp  *.pem ~/Development/chef-repo/.chef
$ cp knife.rb ~/Development/chef-repo/.chef

Let's check if we can validate against the cheff server with the knife tool. If at this point the command fails, means that we have wrong certificates or with wrong name:

$ knife client list andreu2-validator

Cool. Now let's download some community cookbooks, like apache2 and networking_basic:

$ knife cookbook site install apache2
Installing apache2 to /home/amartin/Development/chef-repo/cookbooks
Checking out the master branch.
Creating pristine copy branch chef-vendor-apache2
Downloading apache2 from the cookbooks site at version 1.6.0 to /home/amartin/Development/chef-repo/cookbooks/apache2.tar.gz
Cookbook saved: /home/amartin/Development/chef-repo/cookbooks/apache2.tar.gz
Removing pre-existing version.
Uncompressing apache2 version 1.6.0.
removing downloaded tarball
1 files updated, committing changes
Creating tag cookbook-site-imported-apache2-1.6.0
Checking out the master branch.
Updating b5a1d0d..3eb507c
\
[...]
$ knife cookbook site install networking_basic
Installing networking_basic to /home/amartin/Development/chef-repo/cookbooks
Checking out the master branch.
Creating pristine copy branch chef-vendor-networking_basic
Downloading networking_basic from the cookbooks site at version 0.0.5 to /home/amartin/Development/chef-repo/cookbooks/networking_basic.tar.gz
Cookbook saved: /home/amartin/Development/chef-repo/cookbooks/networking_basic.tar.gz
Removing pre-existing version.
Uncompressing networking_basic version 0.0.5.
removing downloaded tarball
1 files updated, committing changes
Creating tag cookbook-site-imported-networking_basic-0.0.5
Checking out the master branch.
Updating 3eb507c..8f091db
[...]
Now we have both cookbooks inside the folder 'cookbooks'. If you have a look, you can see they are composed by recipes with different functions. For example, let's review this one:

$ cat cookbooks/apache2/recipes/mod_perl.rb
#
# Cookbook Name:: apache2
# Recipe:: perl
#
# adapted from the mod_python recipe by Jeremy Bingham
#
# Copyright 2008-2009, Opscode, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

case node['platform_family']
when "debian"
  %w{libapache2-mod-perl2 libapache2-request-perl apache2-mpm-prefork}.each do |pkg|

    package pkg

  end
when "rhel", "fedora"

  package "mod_perl" do
    notifies :run, "execute[generate-module-list]", :immediately
  end

  package "perl-libapreq2"

end

file "#{node['apache']['dir']}/conf.d/perl.conf" do
  action :delete
  backup false
end

apache_module "perl"

We can appreciate some basic instructions for package download on both Debian and Redhat, also module activation. 

Let's create our own Cookbook. First install magic_shell:

$ knife cookbook site install magic_shell
Installing magic_shell to /home/amartin/Development/chef-repo/cookbooks
Now we can create a cookbook, for example 'myalias':
$ knife cookbook create myalias
** Creating cookbook myalias
** Creating README for cookbook: myalias
** Creating metadata for cookbook: myalias
Now we will add dependencies, to use magic_shell at least our current version:

$ vim cookbooks/myalias/metadata.rb

Add the line depends          'magic_shell', '~> 0.2.0' at the end. Save the file.

Now we can create our own recipe. Let's edit the file cookbooks/myalias/recipes/default.rb and add a few alias and environment variables:

#
# Cookbook Name:: myalias
# Recipe:: default
#
# Copyright 2013, YOUR_COMPANY_NAME
#
# All rights reserved - Do Not Redistribute
#
# Alias `rm` made secured
magic_shell_alias 'rm' do
  command 'rm -i'
end

# Alias cow says mooo
magic_shell_alias 'cow' do
  command 'echo cow says moooo'
end

# Environmental settings, my editor is vim
magic_shell_environment 'EDITOR' do
  value 'vim'
end

Now we upload the cookbook to our company. Since we are not using apache2 or networking here, we can delete their cookbooks:

$ knife cookbook delete apache2
Do you really want to delete apache2 version 1.6.0? (Y/N) y
$ knife cookbook delete networking_basic
Do you really want to delete networking_basic version 0.0.5? (Y/N) y

$ rm -fr cookbooks/apache2 cookbooks/networking_basic

$ knife cookbook upload -a
Uploading magic_shell                  [0.2.0]
Uploading myalias                      [0.0.1]
Uploaded 2 cookbooks.

Now the official manual advises how to deploy virtual machines with vagrant and virtualbox applying cookbooks - is a must read!. However I will apply this changes to my local machine to speed up. First, create a user chef (for example) and we add him to sudoers:

$ sudo useradd -m chef
$ sudo passwd chef
$ sudo visudo (add chef to sudoers)

Now we confirm what is our node name:

$ knife node list
andreuantonio

And now we make effective our new cookbook. -N is for node name, and --sudo is to make clear sudo is needed to change the system:

$ knife bootstrap my_node_server   -N andreuantonio --ssh-user chef   --ssh-password cheffpassword   --ssh-port 22   --run-list "recipe[myalias]" --sudo
Bootstrapping Chef on my_node_server
my_node_server knife sudo password:
Enter your password:

my_node_server [2013-03-27T15:54:11+08:00] INFO: Setting the run_list to ["recipe[myalias]"] from JSON
my_node_server [2013-03-27T15:54:11+08:00] INFO: Run List is [recipe[myalias]]
my_node_server [2013-03-27T15:54:11+08:00] INFO: Run List expands to [myalias]
my_node_server [2013-03-27T15:54:11+08:00] INFO: Starting Chef Run for andreuantonio
my_node_server [2013-03-27T15:54:11+08:00] INFO: Running start handlers
my_node_server [2013-03-27T15:54:11+08:00] INFO: Start handlers complete.
my_node_server [2013-03-27T15:54:13+08:00] INFO: Loading cookbooks [magic_shell, myalias]
my_node_server [2013-03-27T15:54:15+08:00] INFO: Storing updated cookbooks/magic_shell/resources/environment.rb in the cache.
my_node_server [2013-03-27T15:54:16+08:00] INFO: Storing updated cookbooks/magic_shell/resources/alias.rb in the cache.
my_node_server [2013-03-27T15:54:18+08:00] INFO: Storing updated cookbooks/magic_shell/providers/environment.rb in the cache.
my_node_server [2013-03-27T15:54:19+08:00] INFO: Storing updated cookbooks/magic_shell/providers/alias.rb in the cache.
my_node_server [2013-03-27T15:54:20+08:00] INFO: Storing updated cookbooks/magic_shell/Rakefile in the cache.
my_node_server [2013-03-27T15:54:22+08:00] INFO: Storing updated cookbooks/magic_shell/metadata.rb in the cache.
my_node_server [2013-03-27T15:54:23+08:00] INFO: Storing updated cookbooks/magic_shell/CHANGELOG.md in the cache.
my_node_server [2013-03-27T15:54:24+08:00] INFO: Storing updated cookbooks/magic_shell/README.md in the cache.
my_node_server [2013-03-27T15:54:26+08:00] INFO: Storing updated cookbooks/magic_shell/.travis.yml in the cache.
my_node_server [2013-03-27T15:54:27+08:00] INFO: Storing updated cookbooks/magic_shell/.gitignore in the cache.
my_node_server [2013-03-27T15:54:29+08:00] INFO: Storing updated cookbooks/magic_shell/metadata.json in the cache.
my_node_server [2013-03-27T15:54:30+08:00] INFO: Storing updated cookbooks/magic_shell/.rvmrc in the cache.
my_node_server [2013-03-27T15:54:32+08:00] INFO: Storing updated cookbooks/myalias/recipes/default.rb in the cache.
my_node_server [2013-03-27T15:54:33+08:00] INFO: Storing updated cookbooks/myalias/README.md in the cache.
my_node_server [2013-03-27T15:54:34+08:00] INFO: Storing updated cookbooks/myalias/metadata.rb in the cache.
my_node_server [2013-03-27T15:54:34+08:00] INFO: Processing magic_shell_alias[rm] action add (myalias::default line 10)
my_node_server [2013-03-27T15:54:34+08:00] INFO: Adding rm.sh to /etc/profile.d/
my_node_server [2013-03-27T15:54:34+08:00] INFO: Processing file[/etc/profile.d/rm.sh] action create (/var/chef/cache/cookbooks/magic_shell/providers/alias.rb line 7)
my_node_server [2013-03-27T15:54:34+08:00] INFO: file[/etc/profile.d/rm.sh] created file /etc/profile.d/rm.sh
my_node_server [2013-03-27T15:54:34+08:00] INFO: file[/etc/profile.d/rm.sh] mode changed to 755
my_node_server [2013-03-27T15:54:34+08:00] INFO: Processing file[/etc/profile.d/rm.sh] action nothing (/var/chef/cache/cookbooks/magic_shell/providers/alias.rb line 7)
my_node_server [2013-03-27T15:54:34+08:00] INFO: Processing magic_shell_alias[cow] action add (myalias::default line 15)
my_node_server [2013-03-27T15:54:34+08:00] INFO: Adding cow.sh to /etc/profile.d/
my_node_server [2013-03-27T15:54:34+08:00] INFO: Processing file[/etc/profile.d/cow.sh] action create (/var/chef/cache/cookbooks/magic_shell/providers/alias.rb line 7)
my_node_server [2013-03-27T15:54:34+08:00] INFO: file[/etc/profile.d/cow.sh] created file /etc/profile.d/cow.sh
my_node_server [2013-03-27T15:54:34+08:00] INFO: file[/etc/profile.d/cow.sh] mode changed to 755
my_node_server [2013-03-27T15:54:34+08:00] INFO: Processing file[/etc/profile.d/cow.sh] action nothing (/var/chef/cache/cookbooks/magic_shell/providers/alias.rb line 7)
my_node_server [2013-03-27T15:54:34+08:00] INFO: Processing magic_shell_environment[EDITOR] action add (myalias::default line 20)
my_node_server [2013-03-27T15:54:34+08:00] INFO: Adding EDITOR.sh to /etc/profile.d/
my_node_server [2013-03-27T15:54:34+08:00] INFO: Processing file[/etc/profile.d/EDITOR.sh] action create (/var/chef/cache/cookbooks/magic_shell/providers/environment.rb line 7)
my_node_server [2013-03-27T15:54:34+08:00] INFO: file[/etc/profile.d/EDITOR.sh] created file /etc/profile.d/EDITOR.sh
my_node_server [2013-03-27T15:54:34+08:00] INFO: file[/etc/profile.d/EDITOR.sh] mode changed to 755
my_node_server [2013-03-27T15:54:34+08:00] INFO: Processing file[/etc/profile.d/EDITOR.sh] action nothing (/var/chef/cache/cookbooks/magic_shell/providers/environment.rb line 7)
my_node_server [2013-03-27T15:54:37+08:00] INFO: Chef Run complete in 26.190131 seconds
my_node_server [2013-03-27T15:54:37+08:00] INFO: Running report handlers
my_node_server [2013-03-27T15:54:37+08:00] INFO: Report handlers complete
Done! now if we login trough ssh we can find out the changes:


$ ssh chef@my_node_server
chef@my_node_server's password:
Last login: Wed Mar 27 16:04:49 2013 from XXXXXX
$ cow
cow says moooo

Of course, that changes are for the whole system not just for chef user :)

Now we can have our system updating changes using the chef-client daemon. The configuration file resides in /etc/chef/client.rb, make a copy first as client.rb.orig

Now you can customize the configuration file, mine's is a simple one with this parameters:
$ sudo cat client.rb
log_level        :info
log_location     STDOUT
chef_server_url  "https://api.opscode.com/organizations/andreu2"
validation_client_name "andreu2-validator"
node_name "andreuantonio"

We copy the files andreuantonio.pem and andreu2-validator.pem to /etc/chef/ and we can start the daemon. We can see it working in the log file:
/var/log/chef$ cat client.log
[2013-03-27T16:05:43+08:00] INFO: *** Chef 10.12.0 ***
[2013-03-27T16:05:46+08:00] INFO: Run List is [recipe[myalias]]
[2013-03-27T16:05:46+08:00] INFO: Run List expands to [myalias]
[2013-03-27T16:05:46+08:00] INFO: Starting Chef Run for andreuantonio
[2013-03-27T16:05:46+08:00] INFO: Running start handlers
[2013-03-27T16:05:46+08:00] INFO: Start handlers complete.
[2013-03-27T16:05:48+08:00] INFO: Loading cookbooks [magic_shell, myalias]
[2013-03-27T16:05:48+08:00] INFO: Processing magic_shell_alias[rm] action add (myalias::default line 10)
[2013-03-27T16:05:48+08:00] INFO: Adding rm.sh to /etc/profile.d/
[2013-03-27T16:05:48+08:00] INFO: Processing file[/etc/profile.d/rm.sh] action create (/var/chef/cache/cookbooks/magic_shell/providers/alias.rb line 7)
[2013-03-27T16:05:48+08:00] INFO: Processing file[/etc/profile.d/rm.sh] action nothing (/var/chef/cache/cookbooks/magic_shell/providers/alias.rb line 7)
[2013-03-27T16:05:48+08:00] INFO: Processing magic_shell_alias[cow] action add (myalias::default line 15)
[2013-03-27T16:05:48+08:00] INFO: Adding cow.sh to /etc/profile.d/
[2013-03-27T16:05:48+08:00] INFO: Processing file[/etc/profile.d/cow.sh] action create (/var/chef/cache/cookbooks/magic_shell/providers/alias.rb line 7)
[2013-03-27T16:05:48+08:00] INFO: Processing file[/etc/profile.d/cow.sh] action nothing (/var/chef/cache/cookbooks/magic_shell/providers/alias.rb line 7)
[2013-03-27T16:05:48+08:00] INFO: Processing magic_shell_environment[EDITOR] action add (myalias::default line 20)
[2013-03-27T16:05:48+08:00] INFO: Adding EDITOR.sh to /etc/profile.d/
[2013-03-27T16:05:48+08:00] INFO: Processing file[/etc/profile.d/EDITOR.sh] action create (/var/chef/cache/cookbooks/magic_shell/providers/environment.rb line 7)
[2013-03-27T16:05:48+08:00] INFO: Processing file[/etc/profile.d/EDITOR.sh] action nothing (/var/chef/cache/cookbooks/magic_shell/providers/environment.rb line 7)
[2013-03-27T16:05:50+08:00] INFO: Chef Run complete in 4.702428 seconds
[2013-03-27T16:05:50+08:00] INFO: Running report handlers
[2013-03-27T16:05:50+08:00] INFO: Report handlers complete


That's all. Please check Opscode website to view all the options and capabilities of this software - it's huge!

 

No comments:

Post a Comment