fbpx

CL LAB

HOME > CL LAB > Chef > Migrating from Knife-Solo to Knife-Zero #getchef

Migrating from Knife-Solo to Knife-Zero #getchef

 ★ 170

This article was posted in more than a year ago. This information may be old, please note.

This is the English translation of Knife-SoloからKnife-Zeroへの移行 (2014/11/14).

Introduction

Last time, we compared Chef-Solo and Chef-Client Local Mode and converged the localhost. This time, we will compare Knife-Solo and Knife-Zero that are used for managing a remote host.

As shown in the illustration below, Knife-Solo transfers policies such as Cookbooks and Attributes that are on the local file system to a remote host using ssh+rsync. Then, it logs into the remote host via ssh, executes “Chef-Solo”, refers to the policies and converges.

knife-solo

On the other hand, Knife-Zero starts up Chef-Zero to refer to policies on the localhost or Node Object. Then, logs into a remote host via ssh while setting up the TCP port forwarding, refers to the policies in Chef-Zero and converges.

knife-zero

Put very simply, Knife-Zero could act like a client-server as if there is a Chef-Server (even when there isn’t one), and converges a remote host by using Chef-Client Local Mode.
Now, let’s look at some examples of Knife-Zero.

Preparations

Prepare two Ubuntu 12.04 LTS: One Ubuntu 12.04 LTS with Chef DK 0.3.2 installed – this will be used as a workstation (ws.example.jp).
The other Ubuntu 12.04 LTS will be used as a remote host (node.example.jp) for converging.

ubuntu@ws:~$ sudo dpkg -i chefdk_0.3.2-1_amd64.deb
以前に未選択のパッケージ chefdk を選択しています。
(データベースを読み込んでいます ... 現在 49086 個のファイルとディレクトリがインストールされています。)
(chefdk_0.3.2-1_amd64.deb から) chefdk を展開しています...
chefdk (0.3.2-1) を設定しています ...
Thank you for installing Chef Development Kit!
ubuntu@ws:~$

On the workstation, generate an SSH key pair without a passphrase and transfer the public key to the remote host in advance.

ubuntu@ws:~$ ssh-keygen -N ''
Generating public/private rsa key pair.
Enter file in which to save the key (/home/ubuntu/.ssh/id_rsa):
Created directory '/home/ubuntu/.ssh'.
Your identification has been saved in /home/ubuntu/.ssh/id_rsa.
Your public key has been saved in /home/ubuntu/.ssh/id_rsa.pub.
The key fingerprint is:
2e:51:77:b0:2e:6b:bd:14:a9:a6:cc:ef:65:27:82:53 ubuntu@ws.example.jp
The key's randomart image is:
+--[ RSA 2048]----+
|          .      |
|           o     |
|        . o .    |
|       . o o     |
|      . E +      |
|       = = .     |
|      + B * .    |
|     o * = +     |
|      +oo .      |
+-----------------+
ubuntu@ws:~$
ubuntu@node:~$ mkdir .ssh
ubuntu@node:~$ cat > .ssh/authorized_keys
ssh-rsa (snip) ubuntu@ws.example.jp
ubuntu@node:~$

The login user “ubuntu” of the remote host must be set in such a way that “sudo” command can be executed without entering a password.

On the workstation, create a chef-repo directory using “chef generate repo” command. We will use this working directory from now on.

ubuntu@ws:~$ chef generate repo chef-repo
Compiling Cookbooks...
Recipe: code_generator::repo
  * directory[/home/ubuntu/chef-repo] action create
    - create new directory /home/ubuntu/chef-repo
    :
    :
    :
    (diff output suppressed by config)
ubuntu@ws:~$

ubuntu@ws:~$ tree -a chef-repo
chef-repo
├── .gitignore
├── LICENSE
├── README.md
├── Rakefile
├── certificates
│   └── README.md
├── chefignore
├── config
│   └── rake.rb
├── cookbooks
│   └── README.md
├── data_bags
│   └── README.md
├── environments
│   └── README.md
└── roles
    └── README.md

6 directories, 11 files
ubuntu@ws:~$

Let’s place the chef-repo under the git management. We will not go into the detailed procedures here. Please refer to our previous post about Chef Starter Kit (Japanese article) for more information.

We will use an ntp Cookbook as an example. The message “WARNING: No knife configuration file found” will appear frequently from now on because “.chef/knife.rb” does not exist. You can ignore the message for now.

ubuntu@ws:~/chef-repo$ knife cookbook site install ntp -o cookbooks
WARNING: No knife configuration file found
Installing ntp to /home/ubuntu/chef-repo/cookbooks
Checking out the master branch.
Creating pristine copy branch chef-vendor-ntp
Downloading ntp from the cookbooks site at version 1.6.5 to /home/ubuntu/chef-repo/cookbooks/ntp.tar.gz
Cookbook saved: /home/ubuntu/chef-repo/cookbooks/ntp.tar.gz
Removing pre-existing version.
Uncompressing ntp version 1.6.5.
removing downloaded tarball
1 files updated, committing changes
Creating tag cookbook-site-imported-ntp-1.6.5
Checking out the master branch.
Updating 4878097..b23a1b8
Fast-forward
    :
    :
    :
Cookbook ntp version 1.6.5 successfully installed
ubuntu@ws:~/chef-repo$

Create a Role to be used for setting.

ubuntu@ws:~/chef-repo$ vi roles/ntp.json
{
  "json_class": "Chef::Role",
  "name": "ntp",
  "description": "ntp basic",
  "default_attributes": {
    "ntp": {
      "servers": [
        "0.ubuntu.pool.ntp.org",
        "1.ubuntu.pool.ntp.org",
        "2.ubuntu.pool.ntp.org",
        "3.ubuntu.pool.ntp.org"
      ]
    }
  },
  "run_list": [
    "recipe[ntp]"
  ]
}
ubuntu@ws:~/chef-repo$

Knife-Solo

Installing Knife-Solo

Install Knife-Solo using “chef gem install” command.

ubuntu@ws:~/chef-repo$ chef gem install knife-solo
Fetching: knife-solo-0.4.2.gem (100%)
WARNING:  You don't have /home/ubuntu/.chefdk/gem/ruby/2.1.0/bin in your PATH,
      gem executables will not run.
Thanks for installing knife-solo!

If you run into any issues please let us know at:
  https://github.com/matschaffer/knife-solo/issues

If you are upgrading knife-solo please uninstall any old versions by
running `gem clean knife-solo` to avoid any errors.

See http://bit.ly/CHEF-3255 for more information on the knife bug
that causes this.
Successfully installed knife-solo-0.4.2
Parsing documentation for knife-solo-0.4.2
Installing ri documentation for knife-solo-0.4.2
Done installing documentation for knife-solo after 1 seconds
1 gem installed
ubuntu@ws:~/chef-repo$

Installing Chef-Solo to the remote host

Use “knife solo prepare” command to install a Chef-Client Omnibus package on the specified host.

ubuntu@ws:~/chef-repo$ knife solo prepare node.example.jp
WARNING: No knife configuration file found
Bootstrapping Chef...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 16472  100 16472    0     0  14504      0  0:00:01  0:00:01 --:--:-- 18570
Downloading Chef 11.16.0 for ubuntu...
    :
    :
    :
chef (11.16.0-1) を設定しています ...
Thank you for installing Chef!
Generating node config './nodes/node.example.jp.json'...
ubuntu@ws:~/chef-repo$

The Chef-Client Omnibus package is installed on the remote host now. At the same time, a JSON file for node setting is generated in the workstation’s nodes/ directory.

ubuntu@ws:~/chef-repo$ cat ./nodes/node.example.jp.json
{
  "run_list": [

  ],
  "automatic": {
    "ipaddress": "node.example.jp"
  }
}
ubuntu@ws:~/chef-repo$

Executing Chef-Solo on the remote host to converge

Edit the JSON file for node setting, and add the previously created Role to the run_list.

ubuntu@ws:~/chef-repo$ vi nodes/node.example.jp.json
ubuntu@ws:~/chef-repo$ git diff
diff --git a/nodes/node.example.jp.json b/nodes/node.example.jp.json
index cf38459..575ca94 100644
--- a/nodes/node.example.jp.json
+++ b/nodes/node.example.jp.json
@@ -1,6 +1,6 @@
 {
   "run_list": [
-
+    "role[ntp]"
   ],
   "automatic": {
     "ipaddress": "node.example.jp"
ubuntu@ws:~/chef-repo$

Use “knife solo cook“ command to transfer the chef-repo to the specified host, and then execute chef-solo on the specified host.

ubuntu@ws:~/chef-repo$ knife solo cook node.example.jp
WARNING: No knife configuration file found
Running Chef on node.example.jp...
Checking Chef version...
Uploading the kitchen...
Generating solo config...
Running Chef...
    :
    :
    :
Starting Chef Client, version 11.16.0
Compiling Cookbooks...
Converging 9 resources
Recipe: ntp::default
  * package[ntp] action install
    - install version 1:4.2.6.p3+dfsg-1ubuntu3.1 of package ntp
  * package[ntpdate] action install (up to date)
  * directory[/var/lib/ntp] action create (up to date)
  * directory[/var/log/ntpstats/] action create (up to date)
  * cookbook_file[/etc/ntp.leapseconds] action create
    - create new file /etc/ntp.leapseconds
    - update content in file /etc/ntp.leapseconds from none to 274665
    :
    :
    :
    - change mode from '' to '0644'
    - change owner from '' to 'root'
    - change group from '' to 'root'
Recipe: ntp::apparmor
  * service[apparmor] action nothing (skipped due to action :nothing)
  * cookbook_file[/etc/apparmor.d/usr.sbin.ntpd] action create
    - update content in file /etc/apparmor.d/usr.sbin.ntpd from a88a6b to fdf13c
    :
    :
    :
Recipe: ntp::default
  * template[/etc/ntp.conf] action create
    - update content in file /etc/ntp.conf from 4eb9a0 to 64d24c
    :
    :
    :
  * service[ntp] action enable (up to date)
  * service[ntp] action start (up to date)
Recipe: ntp::apparmor
  * service[apparmor] action restart
    - restart service service[apparmor]
Recipe: ntp::default
  * service[ntp] action restart
    - restart service service[ntp]

Running handlers:
Running handlers complete
Chef Client finished, 6/11 resources updated in 14.870581634 seconds
ubuntu@ws:~/chef-repo$

Make sure that the specified policy has been applied.

ubuntu@ws:~/chef-repo$ ssh node.example.jp ps auxwwwf | grep '[ n]tp'
ntp       9914  0.0  0.0  37780  2192 ?        Ss   16:52   0:00 /usr/sbin/ntpd -p /var/run/ntpd.pid -g -u 106:113
ubuntu@ws:~/chef-repo$

ubuntu@ws:~/chef-repo$ ssh node.example.jp grep pool.ntp.org /etc/ntp.conf
server 0.ubuntu.pool.ntp.org iburst
restrict 0.ubuntu.pool.ntp.org nomodify notrap noquery
server 1.ubuntu.pool.ntp.org iburst
restrict 1.ubuntu.pool.ntp.org nomodify notrap noquery
server 2.ubuntu.pool.ntp.org iburst
restrict 2.ubuntu.pool.ntp.org nomodify notrap noquery
server 3.ubuntu.pool.ntp.org iburst
restrict 3.ubuntu.pool.ntp.org nomodify notrap noquery
ubuntu@ws:~/chef-repo$

Converging is completed successfully.

Now, let’s look at the files transferred to the remote host.

ubuntu@ws:~/chef-repo$ ssh node.example.jp tree -aF -L 2 chef-solo
chef-solo
├── cookbooks-1/
│   └── chef-solo-search/
├── cookbooks-2/
│   ├── README.md
│   └── ntp/
├── data_bags/
│   └── README.md
├── dna.json
├── environments/
│   └── README.md
├── nodes/
│   └── node.example.jp.json
├── roles/
│   ├── README.md
│   └── ntp.json
└── solo.rb

8 directories, 8 files
ubuntu@ws:~/chef-repo$

As you can see, Knife-Solo transfers chef-repo to the remote host with rsync, and executes Chef-Solo on the remote host.
If rsync is not installed on the remote host, transferring of the chef-repo fails.

ubuntu@ws:~/chef-repo$ knife solo cook node.example.jp
WARNING: No knife configuration file found
Running Chef on node.example.jp...
Checking Chef version...
Uploading the kitchen...
bash: rsync: command not found
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(605) [sender=3.0.9]
ERROR: RuntimeError: Failed to launch command ["rsync", (snip)

Knife-Zero

Preparations

Before applying policy with Knife-Zero, uninstall the ntp package and chef package that have been installed on the remote host. Even if they are remained on the host, they do not have any impact on the operations. However, it is better to uninstall them in advance to check the operations of Knife-Zero clearly.

ubuntu@ws:~/chef-repo$ ssh node.example.jp sudo apt-get purge ntp chef -y
パッケージリストを読み込んでいます...
依存関係ツリーを作成しています...
状態情報を読み取っています...
以下のパッケージが自動でインストールされましたが、もう必要とされていません:
  libopts25 libcap2
これらを削除するには 'apt-get autoremove' を利用してください。
以下のパッケージは「削除」されます:
  chef* ntp*
アップグレード: 0 個、新規インストール: 0 個、削除: 2 個、保留: 6 個。
この操作後に 111MB のディスク容量が解放されます。
(データベースを読み込んでいます ... 現在 61148 個のファイルとディレクトリがインストールされています。)
chef を削除しています ...
chef の設定ファイルを削除しています ...
ntp を削除しています ...
 * Stopping NTP server ntpd
   ...done.
ntp の設定ファイルを削除しています ...
man-db のトリガを処理しています ...
ureadahead のトリガを処理しています ...
ubuntu@ws:~/chef-repo$

Delete the JSON file for node setting, which was generated after executing Knife-Solo.

ubuntu@ws:~/chef-repo$ rm nodes/node.example.jp.json
ubuntu@ws:~/chef-repo$

Installing Knife-Zero

Install Knife-Zero using “chef gem install” command.

ubuntu@ws:~/chef-repo$ chef gem install knife-zero
Fetching: knife-zero-1.0.0.gem (100%)
WARNING:  You don't have /home/ubuntu/.chefdk/gem/ruby/2.1.0/bin in your PATH,
      gem executables will not run.
Successfully installed knife-zero-1.0.0
Parsing documentation for knife-zero-1.0.0
Installing ri documentation for knife-zero-1.0.0
Done installing documentation for knife-zero after 0 seconds
1 gem installed
ubuntu@ws:~/chef-repo$

Installing Chef-Client to the remote host

Use “knife zero bootstrap” command to install a Chef-Client Omnibus package to the specified host. Use “-x“ option to specify a login user, and “--sudo” option enables to get a root privilege.

ubuntu@ws:~/chef-repo$ knife zero bootstrap node.example.jp -x ubuntu --sudo
WARNING: No knife configuration file found
Connecting to node.example.jp
node.example.jp Installing Chef Client...
    :
    :
    :
node.example.jp chef (11.16.4-1) を設定しています ...
node.example.jp Thank you for installing Chef!
node.example.jp Starting first Chef Client run...
    :
    :
    :
node.example.jp Starting Chef Client, version 11.16.4
node.example.jp Creating a new client identity for node.example.jp using the validator key.
node.example.jp resolving cookbooks for run list: []
node.example.jp Synchronizing Cookbooks:
node.example.jp Compiling Cookbooks...
node.example.jp [2014-11-11T17:43:21+09:00] WARN: Node node.example.jp has an empty run list.
node.example.jp Converging 0 resources
node.example.jp
node.example.jp Running handlers:
node.example.jp Running handlers complete
node.example.jp Chef Client finished, 0/0 resources updated in 1.116895026 seconds
ubuntu@ws:~/chef-repo$

The Chef-Client Omnibus package is now installed to the remote host. At the same time, “Node Object” is generated in the workstation nodes/ directory.

ubuntu@ws:~/chef-repo$ ls -lF nodes/
合計 32
-rw-rw-r-- 1 ubuntu ubuntu 31250 1111 17:43 node.example.jp.json
ubuntu@ws:~/chef-repo$
ubuntu@ws:~/chef-repo$ view nodes/node.example.jp.json
{
  "name": "node.example.jp",
  "normal": {
    "tags": [

    ]
  },
  "automatic": {
    "network": {
      "interfaces": {
        "lo": {
          "mtu": "16436",
          "flags": [
            "LOOPBACK",
            "UP",
            "LOWER_UP"
          ],
    :
    :
    :
    "recipes": [

    ],
    "roles": [

    ]
  }
}
ubuntu@ws:~/chef-repo$

These objects were generated by the Chef-Zero server, which had been executed on the localhost. Knife-Zero achieves this functionality by linking between the Chef-Zero server executed on the localhost and the Chef-Client executed on the remote host with SSH port forwarding. Unlike Knife-Solo, rsync is not necessary.

As described in the previous post “Migrating from Chef-Solo to Chef Client Local Mode” (Japanese article), these objects can be viewed and operated by executing a knife subcommand with “-z (--local-mode)” option.

ubuntu@ws:~/chef-repo$ knife client list -z
WARNING: No knife configuration file found
node.example.jp
ubuntu@ws:~/chef-repo$

ubuntu@ws:~/chef-repo$ knife node list -z
WARNING: No knife configuration file found
node.example.jp
ubuntu@ws:~/chef-repo$

Executing Chef-Client on the remote host to converge

Edit a Role in advance to see the changes in the file set by Knife-Solo.

ubuntu@ws:~/chef-repo$ vi roles/ntp.json
ubuntu@ws:~/chef-repo$ git diff
diff --git a/roles/ntp.json b/roles/ntp.json
index 045d224..adbc3bc 100644
--- a/roles/ntp.json
+++ b/roles/ntp.json
@@ -5,10 +5,10 @@
   "default_attributes": {
     "ntp": {
       "servers": [
-        "0.ubuntu.pool.ntp.org",
-        "1.ubuntu.pool.ntp.org",
-        "2.ubuntu.pool.ntp.org",
-        "3.ubuntu.pool.ntp.org"
+        "0.asia.pool.ntp.org",
+        "1.asia.pool.ntp.org",
+        "2.asia.pool.ntp.org",
+        "3.asia.pool.ntp.org"
       ]
     }
   },
ubuntu@ws:~/chef-repo$

Add a Role to run_list using a knife command (local mode).

ubuntu@ws:~/chef-repo$ knife node run_list add node.example.jp 'role[ntp]' -z
WARNING: No knife configuration file found
node.example.jp:
  run_list: role[ntp]
ubuntu@ws:~/chef-repo$
ubuntu@ws:~/chef-repo$ git diff nodes/node.exmple.jp.json
diff --git a/nodes/node.example.jp.json b/nodes/node.example.jp.json
index eb7f9b5..c3f0e8a 100644
--- a/nodes/node.example.jp.json
+++ b/nodes/node.example.jp.json
@@ -1321,5 +1321,8 @@
     "roles": [
 
     ]
-  }
+  },
+  "run_list": [
+    "role[ntp]"
+  ]
 }
ubuntu@ws:~/chef-repo$

You can also directly edit the JSON file, which is equivalent to Node Object.

Use “knife zero converge” command to execute Chef-Client on the specified host and converge.

ubuntu@ws:~/chef-repo$ knife zero converge 'name:node.example.jp' -x ubuntu --sudo
WARNING: No knife configuration file found
node.example.jp [2014-11-11T18:23:06+09:00] WARN:
    :
    :
    :
node.example.jp Starting Chef Client, version 11.16.4
node.example.jp resolving cookbooks for run list: ["ntp"]
node.example.jp Synchronizing Cookbooks:
node.example.jp   - ntp
node.example.jp Compiling Cookbooks...
node.example.jp Converging 9 resources
node.example.jp Recipe: ntp::default
node.example.jp   * package[ntp] action install
node.example.jp     - install version 1:4.2.6.p3+dfsg-1ubuntu3.1 of package ntp
node.example.jp   * package[ntpdate] action install (up to date)
node.example.jp   * directory[/var/lib/ntp] action create (up to date)
node.example.jp   * directory[/var/log/ntpstats/] action create (up to date)
node.example.jp   * cookbook_file[/etc/ntp.leapseconds] action create (up to date)
node.example.jp Recipe: ntp::apparmor
node.example.jp   * service[apparmor] action nothing (skipped due to action :nothing)
node.example.jp   * cookbook_file[/etc/apparmor.d/usr.sbin.ntpd] action create
node.example.jp     - update content in file /etc/apparmor.d/usr.sbin.ntpd from a88a6b to fdf13c
    :
    :
    :
node.example.jp Recipe: ntp::default
node.example.jp   * template[/etc/ntp.conf] action create
node.example.jp     - update content in file /etc/ntp.conf from 4eb9a0 to 565516
    :
    :
    :
node.example.jp   * service[ntp] action enable (up to date)
node.example.jp   * service[ntp] action start (up to date)
node.example.jp Recipe: ntp::apparmor
node.example.jp   * service[apparmor] action restart
node.example.jp     - restart service service[apparmor]
node.example.jp Recipe: ntp::default
node.example.jp   * service[ntp] action restart
node.example.jp     - restart service service[ntp]
node.example.jp
node.example.jp Running handlers:
node.example.jp Running handlers complete
node.example.jp Chef Client finished, 5/11 resources updated in 10.573336982 seconds
ubuntu@ws:~/chef-repo$

Make sure that the specified policy has been applied.

ubuntu@ws:~/chef-repo$ knife ssh 'name:node.example.jp' -z ps auxwwwf | grep '[ n]tp'
WARNING: No knife configuration file found
node.example.jp ntp      24654  0.0  0.0  37780  2180 ?        Ss   18:23   0:00 /usr/sbin/ntpd -p /var/run/ntpd.pid -g -u 106:113
ubuntu@ws:~/chef-repo$
ubuntu@ws:~/chef-repo$ knife ssh 'name:node.example.jp' -z grep pool.ntp.org /etc/ntp.conf
WARNING: No knife configuration file found
node.example.jp server 0.asia.pool.ntp.org iburst
node.example.jp restrict 0.asia.pool.ntp.org nomodify notrap noquery
node.example.jp server 1.asia.pool.ntp.org iburst
node.example.jp restrict 1.asia.pool.ntp.org nomodify notrap noquery
node.example.jp server 2.asia.pool.ntp.org iburst
node.example.jp restrict 2.asia.pool.ntp.org nomodify notrap noquery
node.example.jp server 3.asia.pool.ntp.org iburst
node.example.jp restrict 3.asia.pool.ntp.org nomodify notrap noquery
ubuntu@ws:~/chef-repo$

Converging is completed successfully.

Node Object in chef-repo is also updated.

ubuntu@ws:~/chef-repo$ git diff
diff --git a/nodes/node.example.jp.json b/nodes/node.example.jp.json
index eb7f9b5..2cdb327 100644
--- a/nodes/node.example.jp.json
+++ b/nodes/node.example.jp.json
@@ -5,6 +5,42 @@
 
     ]
   },
+  "default": {
+    "ntp": {
+      "servers": [
+        "0.asia.pool.ntp.org",
+        "1.asia.pool.ntp.org",
+        "2.asia.pool.ntp.org",
+        "3.asia.pool.ntp.org"
+      ],
    :
    :
    :
@@ -1316,10 +1365,15 @@
       }
     },
     "recipes": [
-
+      "ntp",
+      "ntp::default",
+      "ntp::apparmor"
     ],
     "roles": [
-
+      "ntp"
     ]
-  }
+  },
+  "run_list": [
+    "role[ntp]"
+  ]
 }
ubuntu@ws:~/chef-repo$

Converging has been completed now on the remote host as if Chef-Server exists. knife zero converge uses Search functionality, and it is possible to converge either in serial or parallel for multiple hosts.

Summary

We have briefly introduced the difference between Knife-Solo and Knife-Zero, and how to migrate and use them. Knife-Zero allows us to use Chef ecosystem without Chef-Server, but through the use of the client-server model recommended by Chef. The migration work to Chef-Server is relatively easy. One of the scenarios you can consider is to start small with Knife-Zero or Hosted Enterprise Chef, and then migrating to Hosted/On-premises Enterprise Chef as the business grows. We hope that this information would help you utilize Knife-Zero.


CL LAB Mail Magazine

CL LABの情報を逃さずチェックしよう!

メールアドレスを登録すると記事が投稿されるとメールで通知します。

メールアドレス: 登録

※登録後メールに記載しているリンクをクリックして認証してください。

Related post