DRBDを用いたHA構成のChef Serverクラスタのプライマリ・セカンダリ移行 #getchef

前回、DRBDを用いたHA構成のChef Serverクラスタを構築しました。
本稿では実例として、プライマリ・セカンダリのGraceful Transitions(ゆるやかな移行)を行ってみます。

■前提

環境

• 環境: QEMU+KVM仮想環境
• パブリックネットワーク: 192.168.122.0/24
• プライベートネットワーク: 192.168.100.0/24
• フロントエンド: 1台
• バックエンド: 2台
• ロードバランサ: フロントエンド1台のため不要
• 名前解決: 全マシンの/etc/hostsに記載 (参考:Hostnames)
• ファイアウォール: テストのため全解放 (参考:Firewalls)
• インストールしたChef Server: chef-server-core-12.0.1-1.x86_64.rpm

フロントエンド

• IPアドレス: 192.168.122.101

• ホスト名: fe1.example.jp

• OS: RHEL6.5

• メモリ: 4GB

• ディスク: 10GB (qcow2)

• フロントエンドの共有ホスト名: chef-server.example.jp

バックエンド

• パブリックIPアドレス: 192.168.122.111

• プライベートIPアドレス: 192.168.100.111

• ホスト名: be1.example.jp

• OS: RHEL6.5

• メモリ: 8GB

• ディスク: 10GB (qcow2) + 5GB (qcow2)

• パブリックIPアドレス: 192.168.122.112

• プライベートIPアドレス: 192.168.100.112

• ホスト名: be2.example.jp

• OS: RHEL6.5

• メモリ: 8GB

• ディスク: 10GB (qcow2) + 5GB (qcow2)

• 共有する仮想IPアドレス: 192.168.122.110

• ホスト名: be.example.jp

■事前の状態確認

まず、バックエンドサーバの状態を確認しておきます。

be1.example.jp (プライマリ)

[redhat@be1 ~]$ sudo chef-server-ctl status
run: bookshelf: (pid 1636) 201s, normally down; run: log: (pid 697) 233s
run: keepalived: (pid 711) 233s; run: log: (pid 704) 233s
run: nginx: (pid 1672) 200s, normally down; run: log: (pid 703) 233s
run: oc_bifrost: (pid 1682) 198s, normally down; run: log: (pid 699) 233s
run: oc_id: (pid 1712) 196s, normally down; run: log: (pid 705) 233s
run: opscode-erchef: (pid 1723) 194s, normally down; run: log: (pid 700) 233s
run: opscode-expander: (pid 1740) 192s, normally down; run: log: (pid 706) 233s
run: opscode-expander-reindexer: (pid 1758) 189s, normally down; run: log: (pid 698) 233s
run: opscode-solr4: (pid 1782) 187s, normally down; run: log: (pid 707) 233s
run: postgresql: (pid 1805) 185s, normally down; run: log: (pid 702) 233s
run: rabbitmq: (pid 1830) 184s, normally down; run: log: (pid 709) 233s
run: redis_lb: (pid 2146) 168s, normally down; run: log: (pid 701) 233s
[redhat@be1 ~]$

全サービスが起動しています。

[redhat@be1 ~]$ sudo chef-server-ctl ha-status
[OK] keepalived HA services enabled.
[OK] cluster status = master
[OK] found VIP IP address and I am master
[OK] found VRRP communications interface eth0
[OK] bookshelf is running correctly, and I am master.
[OK] keepalived is running.
[OK] nginx is running correctly, and I am master.
[OK] oc_bifrost is running correctly, and I am master.
[OK] oc_id is running correctly, and I am master.
[OK] opscode-chef-mover is running.
[OK] opscode-erchef is running correctly, and I am master.
[OK] opscode-expander is running correctly, and I am master.
[OK] opscode-expander-reindexer is running correctly, and I am master.
[OK] opscode-solr4 is running correctly, and I am master.
[OK] postgresql is running correctly, and I am master.
[OK] rabbitmq is running correctly, and I am master.
[OK] redis_lb is running correctly, and I am master.
[OK] all checks passed.
[redhat@be1 ~]$

状態はマスター(cluster status = master)で、各サービスが動作(is running)しており、マスター(I am master)となっています。

[redhat@be1 ~]$ cat /proc/drbd
version: 8.4.5 (api:1/proto:86-101)
GIT-hash: 1d360bde0e095d495786eaeb2a1ac76888e4db96 build by mockbuild@Build64R6, 2014-08-17 19:26:04
0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----
ns:109744 nr:0 dw:39616 dr:110957 al:18 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
[redhat@be1 ~]$

[redhat@be1 ~]$ drbd-overview
0:pc0/0  Connected Primary/Secondary UpToDate/UpToDate /var/opt/opscode/drbd/data ext4 4.0G 160M 3.6G 5%
[redhat@be1 ~]$

DRBD は対向ノードと接続(Connected)しており、プライマリロール(Primary)で動作し、データが一致して最新の状態(UpToDate)です。

be2.example.jp (セカンダリ)

[redhat@be2 ~]$ sudo chef-server-ctl status
down: bookshelf: 137s; run: log: (pid 697) 137s
run: keepalived: (pid 704) 137s; run: log: (pid 692) 137s
down: nginx: 137s; run: log: (pid 699) 137s
down: oc_bifrost: 137s; run: log: (pid 702) 137s
down: oc_id: 137s; run: log: (pid 693) 137s
down: opscode-erchef: 137s; run: log: (pid 701) 137s
down: opscode-expander: 137s; run: log: (pid 703) 137s
down: opscode-expander-reindexer: 137s; run: log: (pid 694) 137s
down: opscode-solr4: 137s; run: log: (pid 695) 137s
down: postgresql: 137s; run: log: (pid 698) 137s
down: rabbitmq: 137s; run: log: (pid 696) 137s
down: redis_lb: 137s; run: log: (pid 700) 137s
[redhat@be2 ~]$

クラスタの状態を管理する keepalived 以外のサービスが停止しています。

[redhat@be2 ~]$ sudo chef-server-ctl ha-status
[OK] keepalived HA services enabled.
[OK] cluster status = backup
[OK] did not find VIP IP address and I am not master
[OK] found VRRP communications interface eth0
[OK] bookshelf is not running, and I am not master.
[OK] keepalived is running.
[OK] nginx is not running, and I am not master.
[OK] oc_bifrost is not running, and I am not master.
[OK] oc_id is not running, and I am not master.
[OK] opscode-erchef is not running, and I am not master.
[OK] opscode-expander is not running, and I am not master.
[OK] opscode-expander-reindexer is not running, and I am not master.
[OK] opscode-solr4 is not running, and I am not master.
[OK] postgresql is not running, and I am not master.
[OK] rabbitmq is not running, and I am not master.
[OK] redis_lb is not running, and I am not master.
[OK] all checks passed.
[redhat@be2 ~]$

状態はバックアップ(cluster status = backup)で、keepalived 以外の各サービスは停止(is not running)しており、スレーブ(I am not master)となっています。ぱっと見でnotがわかりづらいので注意してください。

[redhat@be2 ~]$ cat /proc/drbd
version: 8.4.5 (api:1/proto:86-101)
GIT-hash: 1d360bde0e095d495786eaeb2a1ac76888e4db96 build by mockbuild@Build64R6, 2014-08-17 19:26:04
0: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r-----
ns:0 nr:109744 dw:109744 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
[redhat@be2 ~]$

[redhat@be2 ~]$ drbd-overview
0:pc0/0  Connected Secondary/Primary UpToDate/UpToDate
[redhat@be2 ~]$

DRBD は対向ノードと接続(Connected)しており、セカンダリロール(Secondary)で動作し、データが一致していて最新の状態(UpToDate)です。

データの確認

ここではアップロード済の Cookbook だけを簡単に確認しておきます。

[redhat@ws chef-repo]$ knife cookbook list
apt               2.6.1
bluepill          2.3.1
build-essential   2.1.3
nginx             2.7.4
ohai              2.0.1
rsyslog           1.13.0
runit             1.5.12
starter           1.0.0
yum               3.5.2
yum-epel          0.6.0
[redhat@ws chef-repo]$

■ゆるやかな移行

では、ゆるやかな移行 (Graceful Transitions) を行ってみましょう。特に障害が起こっていない状況での移行です。

be1.example.jpをプライマリからセカンダリに降格

現在プライマリである be1.example.jp の keepalived を停止します。

[redhat@be1 ~]$ sudo chef-server-ctl stop keepalived
ok: down: keepalived: 0s, normally up
[redhat@be1 ~]$

keepalived を停止するとフェイルオーバーが始まります。現プライマリでは、

• 仮想 IP アドレスを削除
• すべてのサービスの停止
• DRBD のアンマウント
• DRBD のセカンダリ移行

が行われ、現セカンダリではその逆が行われるので、しばらく待ちます。
状況は chef-server-ctl status コマンドや chef-server-ctl ha-status コマンドで確認できます。

移行が完了すると、

[redhat@be1 ~]$ sudo chef-server-ctl status
down: bookshelf: 25s; run: log: (pid 697) 679s
down: keepalived: 44s, normally up; run: log: (pid 704) 679s
down: nginx: 28s; run: log: (pid 703) 680s
down: oc_bifrost: 29s; run: log: (pid 699) 680s
down: oc_id: 31s; run: log: (pid 705) 680s
down: opscode-erchef: 33s; run: log: (pid 700) 680s
down: opscode-expander: 35s; run: log: (pid 706) 680s
down: opscode-expander-reindexer: 37s; run: log: (pid 698) 680s
down: opscode-solr4: 38s; run: log: (pid 707) 680s
down: postgresql: 41s; run: log: (pid 702) 680s
down: rabbitmq: 42s; run: log: (pid 709) 680s
down: redis_lb: 44s; run: log: (pid 701) 680s
[redhat@be1 ~]$

サービスがすべて停止し、

[redhat@be1 ~]$ sudo chef-server-ctl ha-status
[OK] keepalived HA services enabled.
[OK] cluster status = backup
[OK] did not find VIP IP address and I am not master
[OK] found VRRP communications interface eth0
[OK] bookshelf is not running, and I am not master.
[ERROR] keepalived is not running.
[OK] nginx is not running, and I am not master.
[OK] oc_bifrost is not running, and I am not master.
[OK] oc_id is not running, and I am not master.
[OK] opscode-chef-mover is running.
[OK] opscode-erchef is not running, and I am not master.
[OK] opscode-expander is not running, and I am not master.
[OK] opscode-expander-reindexer is not running, and I am not master.
[OK] opscode-solr4 is not running, and I am not master.
[OK] postgresql is not running, and I am not master.
[OK] rabbitmq is not running, and I am not master.
[OK] redis_lb is not running, and I am not master.
[ERROR] ERRORS WERE DETECTED.
[redhat@be1 ~]$

状態がバックアップ(cluster status = backup)となります。

[redhat@be1 ~]$ cat /proc/drbd
version: 8.4.5 (api:1/proto:86-101)
GIT-hash: 1d360bde0e095d495786eaeb2a1ac76888e4db96 build by mockbuild@Build64R6, 2014-08-17 19:26:04
0: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r-----
ns:108304 nr:28516 dw:58996 dr:84705 al:17 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
[redhat@be1 ~]$

[redhat@be1 ~]$ drbd-overview
0:pc0/0  Connected Secondary/Primary UpToDate/UpToDate
[redhat@be1 ~]$

DRBD もセカンダリに降格し、対向ノードはプライマリに昇格しました。

プライマリに昇格した be2.example.jp の確認

be2.example.jp の状態も同様に確認してみます。

[redhat@be2 ~]$ sudo chef-server-ctl status
run: bookshelf: (pid 1902) 144s, normally down; run: log: (pid 697) 798s
run: keepalived: (pid 704) 798s; run: log: (pid 692) 798s
run: nginx: (pid 1937) 143s, normally down; run: log: (pid 699) 798s
run: oc_bifrost: (pid 1947) 142s, normally down; run: log: (pid 702) 798s
run: oc_id: (pid 1978) 140s, normally down; run: log: (pid 693) 798s
run: opscode-erchef: (pid 1989) 137s, normally down; run: log: (pid 701) 798s
run: opscode-expander: (pid 2006) 135s, normally down; run: log: (pid 703) 798s
run: opscode-expander-reindexer: (pid 2025) 133s, normally down; run: log: (pid 694) 798s
run: opscode-solr4: (pid 2049) 130s, normally down; run: log: (pid 695) 798s
run: postgresql: (pid 2074) 129s, normally down; run: log: (pid 698) 798s
run: rabbitmq: (pid 2097) 127s, normally down; run: log: (pid 696) 798s
run: redis_lb: (pid 2419) 111s, normally down; run: log: (pid 700) 798s
[redhat@be2 ~]$

サービスがすべて起動し、

[redhat@be2 ~]$ sudo chef-server-ctl ha-status
[OK] keepalived HA services enabled.
[OK] cluster status = master
[OK] found VIP IP address and I am master
[OK] found VRRP communications interface eth0
[OK] bookshelf is running correctly, and I am master.
[OK] keepalived is running.
[OK] nginx is running correctly, and I am master.
[OK] oc_bifrost is running correctly, and I am master.
[OK] oc_id is running correctly, and I am master.
[OK] opscode-erchef is running correctly, and I am master.
[OK] opscode-expander is running correctly, and I am master.
[OK] opscode-expander-reindexer is running correctly, and I am master.
[OK] opscode-solr4 is running correctly, and I am master.
[OK] postgresql is running correctly, and I am master.
[OK] rabbitmq is running correctly, and I am master.
[OK] redis_lb is running correctly, and I am master.
[OK] all checks passed.
[redhat@be2 ~]$

状態がマスター(cluster status = master)となっています。

[redhat@be2 ~]$ cat /proc/drbd
version: 8.4.5 (api:1/proto:86-101)
GIT-hash: 1d360bde0e095d495786eaeb2a1ac76888e4db96 build by mockbuild@Build64R6, 2014-08-17 19:26:04
0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----
ns:29184 nr:108304 dw:137488 dr:4849 al:16 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
[redhat@be2 ~]$

[redhat@be2 ~]$ drbd-overview
0:pc0/0  Connected Primary/Secondary UpToDate/UpToDate /var/opt/opscode/drbd/data ext4 4.0G 160M 3.6G 5%
[redhat@be2 ~]$

DRBD もプライマリに昇格しています。

be1.example.jp のセカンダリ化

keepalived を起動してセカンダリとしてクラスタに復帰させます。

[redhat@be1 ~]$ sudo chef-server-ctl start keepalived
ok: run: keepalived: (pid 3196) 0s
[redhat@be1 ~]$

[redhat@be1 ~]$ sudo chef-server-ctl status
down: bookshelf: 409s; run: log: (pid 697) 1063s
run: keepalived: (pid 3196) 19s; run: log: (pid 704) 1063s
down: nginx: 411s; run: log: (pid 703) 1063s
down: oc_bifrost: 412s; run: log: (pid 699) 1063s
down: oc_id: 414s; run: log: (pid 705) 1063s
down: opscode-erchef: 416s; run: log: (pid 700) 1063s
down: opscode-expander: 418s; run: log: (pid 706) 1063s
down: opscode-expander-reindexer: 420s; run: log: (pid 698) 1063s
down: opscode-solr4: 421s; run: log: (pid 707) 1063s
down: postgresql: 424s; run: log: (pid 702) 1063s
down: rabbitmq: 425s; run: log: (pid 709) 1063s
down: redis_lb: 427s; run: log: (pid 701) 1063s
[redhat@be1 ~]$

[redhat@be1 ~]$ sudo chef-server-ctl ha-status
[OK] keepalived HA services enabled.
[OK] cluster status = backup
[OK] did not find VIP IP address and I am not master
[OK] found VRRP communications interface eth0
[OK] bookshelf is not running, and I am not master.
[OK] keepalived is running.
[OK] nginx is not running, and I am not master.
[OK] oc_bifrost is not running, and I am not master.
[OK] oc_id is not running, and I am not master.
[OK] opscode-chef-mover is running.
[OK] opscode-erchef is not running, and I am not master.
[OK] opscode-expander is not running, and I am not master.
[OK] opscode-expander-reindexer is not running, and I am not master.
[OK] opscode-solr4 is not running, and I am not master.
[OK] postgresql is not running, and I am not master.
[OK] rabbitmq is not running, and I am not master.
[OK] redis_lb is not running, and I am not master.
[OK] all checks passed.
[redhat@be1 ~]$

keepalived を起動したからといって勝手にマスターになったりはしません。

データの確認

事前に確認しておいたアップロード済 Cookbook のリストが同じであることを確認します。

[redhat@ws chef-repo]$ knife cookbook list
apt               2.6.1
bluepill          2.3.1
build-essential   2.1.3
nginx             2.7.4
ohai              2.0.1
rsyslog           1.13.0
runit             1.5.12
starter           1.0.0
yum               3.5.2
yum-epel          0.6.0
[redhat@ws chef-repo]$

新たに Cookbook をアップロードします。

[redhat@ws chef-repo]$ knife upload cookbooks/apache2
Created cookbooks/apache2
[redhat@ws chef-repo]$

Cookbook がアップロードできたことを確認します。

[redhat@ws chef-repo]$ knife cookbook list
apache2           3.0.0
apt               2.6.1
bluepill          2.3.1
build-essential   2.1.3
nginx             2.7.4
ohai              2.0.1
rsyslog           1.13.0
runit             1.5.12
starter           1.0.0
yum               3.5.2
yum-epel          0.6.0
[redhat@ws chef-repo]$

簡単ですが、ゆるやかな移行の成功を確認できたとしましょう。

■再度ゆるやかな移行を行う

先の移行では be1(プライマリ),be2(セカンダリ) を be1(セカンダリ),be2(プライマリ) としました。
ここでは再度、ゆるやかな移行を行ってみます。

be2.example.jpをプライマリからセカンダリに降格

現在プライマリである be2.example.jp の keepalived を停止します。

[redhat@be2 ~]$ sudo chef-server-ctl stop keepalived
ok: down: keepalived: 0s, normally up
[redhat@be2 ~]$

フェイルオーバーが始まるので、しばらく待ちます。移行が完了すると、

[redhat@be2 ~]$ sudo chef-server-ctl status
down: bookshelf: 52s; run: log: (pid 697) 1825s
down: keepalived: 70s, normally up; run: log: (pid 692) 1825s
down: nginx: 53s; run: log: (pid 699) 1825s
down: oc_bifrost: 54s; run: log: (pid 702) 1825s
down: oc_id: 56s; run: log: (pid 693) 1825s
down: opscode-erchef: 57s; run: log: (pid 701) 1825s
down: opscode-expander: 59s; run: log: (pid 703) 1825s
down: opscode-expander-reindexer: 61s; run: log: (pid 694) 1825s
down: opscode-solr4: 63s; run: log: (pid 695) 1825s
down: postgresql: 66s; run: log: (pid 698) 1825s
down: rabbitmq: 67s; run: log: (pid 696) 1825s
down: redis_lb: 69s; run: log: (pid 700) 1825s
[redhat@be2 ~]$

サービスがすべて停止し、

[redhat@be2 ~]$ sudo chef-server-ctl ha-status
[OK] keepalived HA services enabled.
[OK] cluster status = backup
[OK] did not find VIP IP address and I am not master
[OK] found VRRP communications interface eth0
[OK] bookshelf is not running, and I am not master.
[ERROR] keepalived is not running.
[OK] nginx is not running, and I am not master.
[OK] oc_bifrost is not running, and I am not master.
[OK] oc_id is not running, and I am not master.
[OK] opscode-erchef is not running, and I am not master.
[OK] opscode-expander is not running, and I am not master.
[OK] opscode-expander-reindexer is not running, and I am not master.
[OK] opscode-solr4 is not running, and I am not master.
[OK] postgresql is not running, and I am not master.
[OK] rabbitmq is not running, and I am not master.
[OK] redis_lb is not running, and I am not master.
[ERROR] ERRORS WERE DETECTED.
[redhat@be2 ~]$

状態がバックアップ(cluster status = backup)となります。

[redhat@be2 ~]$ cat /proc/drbd
version: 8.4.5 (api:1/proto:86-101)
GIT-hash: 1d360bde0e095d495786eaeb2a1ac76888e4db96 build by mockbuild@Build64R6, 2014-08-17 19:26:04
0: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r-----
ns:49952 nr:136900 dw:186852 dr:8797 al:23 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
[redhat@be2 ~]$

[redhat@be2 ~]$ drbd-overview
0:pc0/0  Connected Secondary/Primary UpToDate/UpToDate
[redhat@be2 ~]$

DRBD もセカンダリに降格し、対向ノードはプライマリに昇格しました。

プライマリに昇格した be1.example.jp の確認

be1.example.jp の状態も同様に確認してみます。

[redhat@be1 ~]$ sudo chef-server-ctl status
run: bookshelf: (pid 3605) 174s, normally down; run: log: (pid 697) 1951s
run: keepalived: (pid 3196) 907s; run: log: (pid 704) 1951s
run: nginx: (pid 3640) 172s, normally down; run: log: (pid 703) 1951s
run: oc_bifrost: (pid 3650) 171s, normally down; run: log: (pid 699) 1951s
run: oc_id: (pid 3685) 168s, normally down; run: log: (pid 705) 1951s
run: opscode-erchef: (pid 3744) 165s, normally down; run: log: (pid 700) 1951s
run: opscode-expander: (pid 3810) 162s, normally down; run: log: (pid 706) 1951s
run: opscode-expander-reindexer: (pid 3883) 159s, normally down; run: log: (pid 698) 1951s
run: opscode-solr4: (pid 3956) 156s, normally down; run: log: (pid 707) 1951s
run: postgresql: (pid 3991) 153s, normally down; run: log: (pid 702) 1951s
run: rabbitmq: (pid 4064) 152s, normally down; run: log: (pid 709) 1951s
run: redis_lb: (pid 4531) 133s, normally down; run: log: (pid 701) 1951s
[redhat@be1 ~]$

サービスがすべて起動し、

[redhat@be1 ~]$ sudo chef-server-ctl ha-status
[OK] keepalived HA services enabled.
[OK] cluster status = master
[OK] found VIP IP address and I am master
[OK] found VRRP communications interface eth0
[OK] bookshelf is running correctly, and I am master.
[OK] keepalived is running.
[OK] nginx is running correctly, and I am master.
[OK] oc_bifrost is running correctly, and I am master.
[OK] oc_id is running correctly, and I am master.
[OK] opscode-erchef is running correctly, and I am master.
[OK] opscode-expander is running correctly, and I am master.
[OK] opscode-expander-reindexer is running correctly, and I am master.
[OK] opscode-solr4 is running correctly, and I am master.
[OK] postgresql is running correctly, and I am master.
[OK] rabbitmq is running correctly, and I am master.
[OK] redis_lb is running correctly, and I am master.
[OK] all checks passed.
[redhat@be1 ~]$

状態がマスター(cluster status = master)となっています。

[redhat@be1 ~]$ cat /proc/drbd
version: 8.4.5 (api:1/proto:86-101)
GIT-hash: 1d360bde0e095d495786eaeb2a1ac76888e4db96 build by mockbuild@Build64R6, 2014-08-17 19:26:04
0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----
ns:137512 nr:49952 dw:109640 dr:91950 al:19 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
[redhat@be1 ~]$

[redhat@be1 ~]$ drbd-overview
0:pc0/0  Connected Primary/Secondary UpToDate/UpToDate /var/opt/opscode/drbd/data ext4 4.0G 164M 3.6G 5%
[redhat@be1 ~]$

DRBD もプライマリに昇格しています。

be2.example.jp のセカンダリ化

keepalived を起動してセカンダリとしてクラスタに復帰させます。

[redhat@be2 ~]$ sudo chef-server-ctl start keepalived
ok: run: keepalived: (pid 3380) 0s
[redhat@be2 ~]$

[redhat@be2 ~]$ sudo chef-server-ctl status
down: bookshelf: 351s; run: log: (pid 697) 2124s
run: keepalived: (pid 3380) 58s; run: log: (pid 692) 2124s
down: nginx: 352s; run: log: (pid 699) 2124s
down: oc_bifrost: 353s; run: log: (pid 702) 2124s
down: oc_id: 355s; run: log: (pid 693) 2124s
down: opscode-erchef: 356s; run: log: (pid 701) 2124s
down: opscode-expander: 358s; run: log: (pid 703) 2124s
down: opscode-expander-reindexer: 360s; run: log: (pid 694) 2124s
down: opscode-solr4: 362s; run: log: (pid 695) 2124s
down: postgresql: 365s; run: log: (pid 698) 2124s
down: rabbitmq: 366s; run: log: (pid 696) 2124s
down: redis_lb: 368s; run: log: (pid 700) 2124s
[redhat@be2 ~]$

[redhat@be2 ~]$ sudo chef-server-ctl ha-status
[OK] keepalived HA services enabled.
[OK] cluster status = backup
[OK] did not find VIP IP address and I am not master
[OK] found VRRP communications interface eth0
[OK] bookshelf is not running, and I am not master.
[OK] keepalived is running.
[OK] nginx is not running, and I am not master.
[OK] oc_bifrost is not running, and I am not master.
[OK] oc_id is not running, and I am not master.
[OK] opscode-erchef is not running, and I am not master.
[OK] opscode-expander is not running, and I am not master.
[OK] opscode-expander-reindexer is not running, and I am not master.
[OK] opscode-solr4 is not running, and I am not master.
[OK] postgresql is not running, and I am not master.
[OK] rabbitmq is not running, and I am not master.
[OK] redis_lb is not running, and I am not master.
[OK] all checks passed.
[redhat@be2 ~]$

[redhat@be2 ~]$ cat /proc/drbd
version: 8.4.5 (api:1/proto:86-101)
GIT-hash: 1d360bde0e095d495786eaeb2a1ac76888e4db96 build by mockbuild@Build64R6, 2014-08-17 19:26:04
0: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r-----
ns:49952 nr:138108 dw:188060 dr:8797 al:23 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
[redhat@be2 ~]$

[redhat@be2 ~]$ drbd-overview
0:pc0/0  Connected Secondary/Primary UpToDate/UpToDate
[redhat@be2 ~]$

keepalived を起動したからといって勝手にマスターになったりはしません。

データの確認

移行前にアップロードした Cookbook が確かに存在していることを確認します。

[redhat@ws chef-repo]$ knife cookbook list
apache2           3.0.0
apt               2.6.1
bluepill          2.3.1
build-essential   2.1.3
nginx             2.7.4
ohai              2.0.1
rsyslog           1.13.0
runit             1.5.12
starter           1.0.0
yum               3.5.2
yum-epel          0.6.0
[redhat@ws chef-repo]$

■まとめ

本稿では、DRBDを用いたHA構成のChef Serverクラスタにおけるプライマリ・セカンダリのゆるやかな移行を行ってみました。特に障害が起こっていない状態であるため、何も問題はありませんでしたが、普段の何も起きていない状態でどのような処理が行われるかや手順の確認を行い、心構えをしておくとよいでしょう。

✨ このカテゴリーのAIまとめ記事を見る