fbpx

CL LAB

HOME > CL LAB > Chef > Chef Audit Modeを使ってみる #getchef #serverspec

Chef Audit Modeを使ってみる #getchef #serverspec

 ★ 7

この記事は1年以上前に投稿されました。情報が古い可能性がありますので、ご注意ください。

先日リリースされたChef Client 12.1.0にて、「監査モード」(Audit Mode)が実験的に搭載されました。これはNode上でServerspecを実行し、インフラの監査(Audit)を行う機能です。

本稿では、実際のNodeとtest-kitchenでのAudit Modeを見てみます。また、使用したCookbookはcl-lab-k/apache2-take at audit_mode_blogです。

前提条件

  • Workstation : Ubuntu 14.04 LTS, Chef-DK 0.5.0rc5
  • Chef Server : Ubuntu 14.04 LTS, Chef Server 12.0.7
  • Node : Ubuntu 14.04 LTS, Chef Client 12.2.1

実際のNodeでのAudit Mode

最初に、実際のNodeに対してrecipe[apache2-take::defaultを適用しておきます。

このNodeのrun_listにrecipe[apache2-take::audit]を追加します。書式に関して詳しくはRelease Notes: chef-client 12.1 | chef-client, audit-modeを参照してください。

chef-clientに対して「--audit-mode enabled」(収束後に監査を実行)、または「--audit-mode audit-only」(収束せず監査のみを実行)のいずれかのオプションを与えて実行することで、Audit Modeが機能します。

まずは「--audit-mode enabled」を行ってみます。

ubuntu@node1:~$ sudo chef-client --audit-mode enabled
[2015-04-16T13:44:28+09:00] WARN: Chef-client has been configured to audit after it converges. Audit mode is an experimental feature currently under development. API changes may occur. Use at your own risk.
* To enable audit mode after converge, use command line option `--audit-mode enabled` or set `:audit_mode = :enabled` in your config file.
* To disable audit mode, use command line option `--audit-mode disabled` or set `:audit_mode = :disabled` in your config file.
* To only run audit mode, use command line option `--audit-mode audit-only` or set `:audit_mode = :audit_only` in your config file.
Audit mode is disabled by default.
Starting Chef Client, version 12.2.1
resolving cookbooks for run list: ["apache2-take", "apache2-take::audit"]
Synchronizing Cookbooks:
  - apache2-take
Compiling Cookbooks...
Converging 15 resources
Recipe: apache2-take::default
  * execute[apt-get update] action run
    - execute apt-get update
  * apt_package[apache2] action install (up to date)
  * apt_package[git-core] action install (up to date)
  * apt_package[curl] action install (up to date)
  * apt_package[unzip] action install (up to date)
  * service[apache2] action enable (up to date)
  * template[/etc/apache2/ports.conf] action create (up to date)
  * template[/etc/apache2/sites-available/default.conf] action create (up to date)
  * execute[a2ensite default] action run
    - execute a2ensite default
  * git[/var/chef/cache/apache2-take-sample-page] action sync (up to date)
  * execute[/var/www/index.html] action run
    - execute cp -f /var/chef/cache/apache2-take-sample-page/index.html /var/www/index.html
  * remote_file[/var/chef/cache/apache2-take-sample-image.zip] action create (up to date)
  * execute[unzip apache2-take-sample-image.zip] action run (skipped due to not_if)
  * execute[/var/www/img] action run (skipped due to not_if)
  * service[apache2-start] action start
    - start service service[apache2-start]
  * service[apache2] action restart
    - restart service service[apache2]

ここまでは通常の収束処理です。ここから引き続き、監査が始まります。

Starting audit phase

apache2-take::audit
  package apache2
    should be installed
  package git-core
    should be installed
  package curl
    should be installed
  package unzip
    should be installed
  service apache2
    should be enabled
    should be running
    should listening 8080
  file /etc/apache2/ports.conf
    should be a file
    should be owned by root
    should be grouped into root
    should be mode 644
    should contain 'NameVirtualHost *:8080'
    should contain 'Listen 8080'
  file /etc/apache2/sites-available/default.conf
    should be a file
    should be owned by root
    should be grouped into root
    should be mode 644
    should contain '<VirtualHost *:8080>'
  symlink default.conf
    should be linked to the available file
  file www content
    should be a file
    should contain welcome banner
  dir www content
    should be a directory

Finished in 0.27375 seconds (files took 0.95518 seconds to load)
22 examples, 0 failures
Auditing complete

Running handlers:
Running handlers complete
Chef Client finished, 5/14 resources updated in 23.763426765 seconds
  22/22 controls succeeded
ubuntu@node1:~$

収束および監査が正常に完了しました。

引き続き、「--audit-mode audit-only」で監査のみを行ってみます。

ubuntu@node1:~$ sudo chef-client --audit-mode audit-only
[2015-04-16T13:49:49+09:00] WARN: Chef-client has been configured to skip converge and only audit. Audit mode is an experimental feature currently under development. API changes may occur. Use at your own risk.
* To enable audit mode after converge, use command line option `--audit-mode enabled` or set `:audit_mode = :enabled` in your config file.
* To disable audit mode, use command line option `--audit-mode disabled` or set `:audit_mode = :disabled` in your config file.
* To only run audit mode, use command line option `--audit-mode audit-only` or set `:audit_mode = :audit_only` in your config file.
Audit mode is disabled by default.
Starting Chef Client, version 12.2.1
resolving cookbooks for run list: ["apache2-take", "apache2-take::audit"]
Synchronizing Cookbooks:
  - apache2-take
Compiling Cookbooks...
Starting audit phase

apache2-take::audit
  package apache2
    should be installed
  package git-core
    should be installed
  package curl
    should be installed
  package unzip
    should be installed
  service apache2
    should be enabled
    should be running
    should listening 8080
  file /etc/apache2/ports.conf
    should be a file
    should be owned by root
    should be grouped into root
    should be mode 644
    should contain 'NameVirtualHost *:8080'
    should contain 'Listen 8080'
  file /etc/apache2/sites-available/default.conf
    should be a file
    should be owned by root
    should be grouped into root
    should be mode 644
    should contain '<VirtualHost *:8080>'
  symlink default.conf
    should be linked to the available file
  file www content
    should be a file
    should contain welcome banner
  dir www content
    should be a directory

Finished in 0.19382 seconds (files took 0.25124 seconds to load)
22 examples, 0 failures
Auditing complete

Running handlers:
Running handlers complete
Chef Client finished, 0/0 resources updated in 1.687719803 seconds
  22/22 controls succeeded
ubuntu@node1:~$

監査のみが行われ、正常に終了しました。

このように、Recipeを適用して収束を行うのと同じような感覚で、Serverspecによる監査を行うことができます。

test-kitchenでのAudit Mode

test-kitchenでもAudit Modeを適用することができます。しかし、2015年4月現在最新のChef-DK 0.4.0に含まれているtest-kitchen 1.3系はAudit Modeを有効にできません(参考: Chef Audit Mode Introduction, [ChefZero,ChefSolo] Support symbol values in solo.rb & client.rb.)。次のようにアルファ版のChef-DK 0.5系(注: 2015年5月、この問題を解消したChef-DK 0.5.0が正式にリリースされています)をインストールするか、

% curl -L https://chef.io/chef/install.sh | sudo bash -s -- -P chefdk -p

次のようにGemfileでtest-kitchenの1.4系を指定してbundle installしてください。

gem 'test-kitchen', '= 1.4.0.rc.1'

さて、ここではServerspec 1系でテストするtest/integration/default/rspecは削除しておいてください。Serverspec 2系でテストするtest/integration/default/serverspecとGemの依存関係に衝突が起きるからです。test/integration/default/batsは残しておいても構いません。

apache2-take Cookbookの.kitchen.ymlは次のようになります。run_listにrecipe[apache2-take::audit]を追加し、provisioner設定のclient_rb: audit_mode: :enabledを追加しています。このaudit_mode: :enabledの部分がChef-DK 0.4.0のtest-kitchen 1.3系ではエラーになってしまいます。

---
driver
:
  name
: vagrant

provisioner
:
  name
: chef_zero
  client_rb
:
    audit_mode
: :enabled

platforms
:
  - name
: ubuntu-14.04
    driver_config
:
      box
: chef/ubuntu-14.04
      require_chef_omnibus
: true
  - name
: ubuntu-12.04
    driver_config
:
      box
: chef/ubuntu-12.04
      require_chef_omnibus
: true
  - name
: ubuntu-10.04
    driver_config
:
      box
: chef/ubuntu-10.04
      require_chef_omnibus
: true

suites
:
  - name
: default
    run_list
:
     - recipe[apache2-take::default]
      - recipe[apache2-take::audit]

これでテストを実行してみましょう。

% bundle exec kitchen test
-----> Starting Kitchen (v1.4.0.rc.1)
-----> Cleaning up any prior instances of <default-ubuntu-1404>
-----> Destroying <default-ubuntu-1404>...
       Finished destroying <default-ubuntu-1404> (0m0.00s).
-----> Testing <default-ubuntu-1404>
-----> Creating <default-ubuntu-1404>...
       Bringing machine 'default' up with 'virtualbox' provider...
       ffi-yajl/json_gem is deprecated, these monkeypatches will be dropped shortly
       ==> default: Importing base box 'chef/ubuntu-14.04'...
==> default: Matching MAC address for NAT networking...
       ==> default: Checking if box 'chef/ubuntu-14.04' is up to date...
       ==> default: Setting the name of the VM: kitchen-apache2-take-default-ubuntu-1404_default_1429156335426_63779
       ==> default: Clearing any previously set network interfaces...
       ==> default: Preparing network interfaces based on configuration...
           default: Adapter 1: nat
       ==> default: Forwarding ports...
           default: 22 => 2222 (adapter 1)
       ==> default: Booting VM...
       ==> default: Waiting for machine to boot. This may take a few minutes...
           default: SSH address: 127.0.0.1:2222
           default: SSH username: vagrant
           default: SSH auth method: private key
           default: Warning: Connection timeout. Retrying...
       ==> default: Machine booted and ready!
       ==> default: Checking for guest additions in VM...
       ==> default: Setting hostname...
       ==> default: Machine not provisioning because `--no-provision` is specified.
       Vagrant instance <default-ubuntu-1404> created.
       Finished creating <default-ubuntu-1404> (0m36.25s).
-----> Converging <default-ubuntu-1404>...
$$$$$$ Running legacy converge for 'Vagrant' Driver
       Preparing files for transfer
       Preparing dna.json
       Resolving cookbook dependencies with Berkshelf 3.2.3...
       Removing non-cookbook files before transfer
       Preparing validation.pem
       Preparing client.rb
       bash: warning: setlocale: LC_ALL: cannot change locale (ja_JP.UTF-8)
-----> Installing Chef Omnibus (install only if missing)
       Downloading https://www.chef.io/chef/install.sh to file /tmp/install.sh
       Trying wget...
       Download complete.
       Downloading Chef  for ubuntu...
       downloading https://www.getchef.com/chef/metadata?v=&prerelease=false&nightlies=false&p=ubuntu&pv=14.04&m=x86_64
         to file /tmp/install.sh.1306/metadata.txt
       trying wget...
       url  https://opscode-omnibus-packages.s3.amazonaws.com/ubuntu/13.04/x86_64/chef_12.2.1-1_amd64.deb
       md5  84119f54115d754373c9891b8759497c
       sha256   8e0a8a2477c11615f86ffe686a68fa6636112ba82ebe6bb22daa5dd416f3c13e
       downloaded metadata file looks valid...
       downloading https://opscode-omnibus-packages.s3.amazonaws.com/ubuntu/13.04/x86_64/chef_12.2.1-1_amd64.deb
         to file /tmp/install.sh.1306/chef_12.2.1-1_amd64.deb
       trying wget...
       Comparing checksum with sha256sum...
       Installing Chef
       installing with dpkg...
       Selecting previously unselected package chef.
(Reading database ... 32400 files and directories currently installed.)
       Preparing to unpack .../chef_12.2.1-1_amd64.deb ...
       Unpacking chef (12.2.1-1) ...
       Setting up chef (12.2.1-1) ...
       Thank you for installing Chef!
       bash: warning: setlocale: LC_ALL: cannot change locale (ja_JP.UTF-8)
       Transferring files to <default-ubuntu-1404>
       bash: warning: setlocale: LC_ALL: cannot change locale (ja_JP.UTF-8)
       [2015-04-16T03:53:27+00:00] WARN: Chef-client has been configured to audit after it converges. Audit mode is an experimental feature currently under development. API changes may occur. Use at your own risk.
       * To enable audit mode after converge, use command line option `--audit-mode enabled` or set `:audit_mode = :enabled` in your config file.
       * To disable audit mode, use command line option `--audit-mode disabled` or set `:audit_mode = :disabled` in your config file.
       * To only run audit mode, use command line option `--audit-mode audit-only` or set `:audit_mode = :audit_only` in your config file.
       Audit mode is disabled by default.
       Starting Chef Client, version 12.2.1
       Creating a new client identity for default-ubuntu-1404 using the validator key.
       [2015-04-16T03:53:34+00:00] WARN: Child with name 'dna.json' found in multiple directories: /tmp/kitchen/dna.json and /tmp/kitchen/dna.json
       [2015-04-16T03:53:34+00:00] WARN: Child with name 'dna.json' found in multiple directories: /tmp/kitchen/dna.json and /tmp/kitchen/dna.json
       resolving cookbooks for run list: ["apache2-take::default", "apache2-take::audit"]
       [2015-04-16T03:53:34+00:00] WARN: Child with name 'dna.json' found in multiple directories: /tmp/kitchen/dna.json and /tmp/kitchen/dna.json
       Synchronizing Cookbooks:
         - apache2-take
       Compiling Cookbooks...
       Converging 15 resources
       Recipe: apache2-take::default
         * execute[apt-get update] action run
           - execute apt-get update
         * apt_package[apache2] action install
           - install version 2.4.7-1ubuntu4.4 of package apache2
         * apt_package[git-core] action install
           - install version 1:1.9.1-1ubuntu0.1 of package git-core
         * apt_package[curl] action install
           - install version 7.35.0-1ubuntu2.3 of package curl
         * apt_package[unzip] action install
           - install version 6.0-9ubuntu1.3 of package unzip
         * service[apache2] action enable (up to date)
         * template[/etc/apache2/ports.conf] action create
           - update content in file /etc/apache2/ports.conf from 9d2d53 to 0ac7d0
           --- /etc/apache2/ports.conf  2014-01-07 13:23:42.000000000 +0000
           +++ /tmp/chef-rendered-template20150416-1408-1mm19yw 2015-04-16 03:54:48.089473477 +0000
           @@ -1,16 +1,3 @@
           -# If you just change the port or add more ports here, you will likely also
           -# have to change the VirtualHost statement in
           -# /etc/apache2/sites-enabled/000-default.conf
           -
           -Listen 80
           -
           -<IfModule ssl_module>
           -    Listen 443
           -</IfModule>
           -
       -<IfModule mod_gnutls.c>
           -    Listen 443
           -</IfModule>
           -
           -# vim: syntax=apache ts=4 sw=4 sts=4 sr noet
           +NameVirtualHost *:8080
           +Listen 8080
         * template[/etc/apache2/sites-available/default.conf] action create
           - create new file /etc/apache2/sites-available/default.conf
           - update content in file /etc/apache2/sites-available/default.conf from none to 850e18
           --- /etc/apache2/sites-available/default.conf    2015-04-16 03:54:48.097473477 +0000
           +++ /tmp/chef-rendered-template20150416-1408-1x8icgv 2015-04-16 03:54:48.097473477 +0000
           @@ -1 +1,18 @@
           +<VirtualHost *:8080>
           +    ServerAdmin webmaster@default-ubuntu-1404
           +    DocumentRoot /var/www
           +    <Directory />
           +        Options FollowSymLinks
            AllowOverride None
           +    </Directory>
           +    <Directory /var/www/>
       +        Options Indexes FollowSymLinks MultiViews
           +        AllowOverride None
           +        Order allow,deny
       
           +    </Directory>
           +    ErrorLog /var/log/apache2/error.log
           +    LogLevel warn
           +    CustomLog /var/log/apache2/access.log combined
           +</VirtualHost>
           - change mode from '' to '0644'
           - change owner from '' to 'root'
           - change group from '' to 'root'
         * execute[a2ensite default] action run
           - execute a2ensite default
         * git[/tmp/kitchen/cache/apache2-take-sample-page] action sync
           - clone from https://github.com/cl-lab-k/apache2-take-sample-page into /tmp/kitchen/cache/apache2-take-sample-page
           - checkout ref 32885d1a22819291c87b2ddd12d8daadd374fcf0 branch HEAD
         * execute[/var/www/index.html] action run
           - execute cp -f /tmp/kitchen/cache/apache2-take-sample-page/index.html /var/www/index.html
         * remote_file[/tmp/kitchen/cache/apache2-take-sample-image.zip] action create
           - create new file /tmp/kitchen/cache/apache2-take-sample-image.zip
           - update content in file /tmp/kitchen/cache/apache2-take-sample-image.zip from none to 0e32c5
           (new content is binary, diff output suppressed)
         * execute[unzip apache2-take-sample-image.zip] action run
           - execute unzip -o -d /tmp/kitchen/cache /tmp/kitchen/cache/apache2-take-sample-image.zip
         * execute[/var/www/img] action run
           - execute mv -f /tmp/kitchen/cache/apache2-take-sample-image-master /var/www/img
         * service[apache2-start] action start
           - start service service[apache2-start]
         * service[apache2] action restart
           - restart service service[apache2]

ここまで通常の収束が行われ、次に監査モードが行われます。

       Starting audit phase
       
       apache2-take::audit
         package apache2
           should be installed
         package git-core
           should be installed
         package curl
           should be installed
         package unzip
           should be installed
         service apache2
           should be enabled
           should be running
           should listening 8080
         file /etc/apache2/ports.conf
           should be a file
           should be owned by root
           should be grouped into root
           should be mode 644
           should contain 'NameVirtualHost *:8080'
           should contain 'Listen 8080'
         file /etc/apache2/sites-available/default.conf
           should be a file
           should be owned by root
           should be grouped into root
           should be mode 644
           should contain '<VirtualHost *:8080>'
         symlink default.conf
           should be linked to the available file
         file www content
           should be a file
           should contain welcome banner
         dir www content
           should be a directory
       
       Finished in 0.23721 seconds (files took 0.41982 seconds to load)
       22 examples, 0 failures
       Auditing complete
       
       Running handlers:
       Running handlers complete
       Chef Client finished, 15/16 resources updated in 104.96236584 seconds
         22/22 controls succeeded
       Finished converging <default-ubuntu-1404> (2m32.17s).

収束と監査が正常に完了し、テストを行います。

-----> Setting up <default-ubuntu-1404>...
$$$$$$ Running legacy setup for 'Vagrant' Driver
       bash: warning: setlocale: LC_ALL: cannot change locale (ja_JP.UTF-8)
-----> Installing Busser (busser)
Fetching: thor-0.19.0.gem (100%)
       Successfully installed thor-0.19.0
Fetching: busser-0.7.0.gem (100%)
       Successfully installed busser-0.7.0
       2 gems installed
-----> Setting up Busser
       Creating BUSSER_ROOT in /tmp/verifier
       Creating busser binstub
       Installing Busser plugins: busser-serverspec
       Plugin serverspec installed (version 0.5.5)
-----> Running postinstall for serverspec plugin
       Finished setting up <default-ubuntu-1404> (0m24.45s).
-----> Verifying <default-ubuntu-1404>...
$$$$$$ Running legacy verify for 'Vagrant' Driver
       Preparing files for transfer
       bash: warning: setlocale: LC_ALL: cannot change locale (ja_JP.UTF-8)
       Suite path directory /tmp/verifier/suites does not exist, skipping.
       Transferring files to <default-ubuntu-1404>
       bash: warning: setlocale: LC_ALL: cannot change locale (ja_JP.UTF-8)
-----> Running serverspec test suite
-----> Bundle Installing..
       /tmp/verifier/gems/gems/busser-serverspec-0.5.5/lib/busser/runner_plugin/serverspec.rb:49:in `run_bundle_install': Use RbConfig instead of obsolete and deprecated Config.
       /tmp/verifier/gems/gems/busser-serverspec-0.5.5/lib/busser/runner_plugin/serverspec.rb:49:in `run_bundle_install'
: Use RbConfig instead of obsolete and deprecated Config.
                run  bundle install --gemfile /tmp/verifier/suites/serverspec/Gemfile --local || bundle install --gemfile /tmp/verifier/suites/serverspec/Gemfile from "."
       Don't run Bundler as root. Bundler can ask for sudo if it is needed, and
       installing your bundle as root will break this application for all non-root
       users on this machine.
       Could not find gem '
serverspec (< 3.0) ruby' in any of the gem sources listed in
       your Gemfile or installed on this machine.
       Don'
t run Bundler as root. Bundler can ask for sudo if it is needed, and
       installing your bundle as root will break this application for all non-root
       users on this machine.
       Fetching gem metadata from https://rubygems.org/.......
       Fetching version metadata from https://rubygems.org/..
       Resolving dependencies...
       Installing diff-lcs 1.2.5
       Installing multi_json 1.11.0
       Installing net-ssh 2.9.2
       Installing net-scp 1.2.1
       Installing rspec-support 3.2.2
       Installing rspec-core 3.2.3
       Installing rspec-expectations 3.2.1
       Installing rspec-mocks 3.2.1
       Installing rspec 3.2.0
       Installing rspec-its 1.2.0
       Installing specinfra 2.28.2
       Installing serverspec 2.14.1
       Using bundler 1.9.4
       Bundle complete! 1 Gemfile dependency, 13 gems now installed.
       Use `bundle show [gemname]` to see where a bundled gem is installed.
       /opt/chef/embedded/bin/ruby -I/tmp/verifier/suites/serverspec -I/tmp/verifier/gems/gems/rspec-support-3.2.2/lib:/tmp/verifier/gems/gems/rspec-core-3.2.3/lib /opt/chef/embedded/bin/rspec --pattern /tmp/verifier/suites/serverspec/\*\*/\*_spec.rb --color --format documentation --default-path /tmp/verifier/suites/serverspec
       
       Package "apache2"
         should be installed
       
       Package "git-core"
         should be installed
       
       Package "curl"
         should be installed
       
       Package "unzip"
         should be installed
       
       Service "apache2"
         should be enabled
         should be running
       
       Port "8080"
         should be listening
       
       File "/etc/apache2/ports.conf"
         should be file
         should be owned by "root"
         should be grouped into "root"
         should be mode 644
       
       File "/etc/apache2/sites-available/default.conf"
         should be file
         should be owned by "root"
         should be grouped into "root"
         should be mode 644
       
       File "/etc/apache2/ports.conf"
         should contain "NameVirtualHost *:8080"
         should contain "Listen 8080"
       
       File "/etc/apache2/sites-available/default.conf"
         should contain "<VirtualHost *:8080>"
       
       File "/etc/apache2/sites-enabled/default.conf"
         should be linked to "../sites-available/default.conf"
       
       File "/var/www/index.html"
         should be file
         should contain "Welcome to my top page"
       
       File "/var/www/img"
         should be directory
       
       Finished in 0.14738 seconds (files took 0.30877 seconds to load)
       22 examples, 0 failures
       
       Finished verifying <default-ubuntu-1404> (0m24.45s).
-----> Destroying <default-ubuntu-1404>...
       ==> default: Forcing shutdown of VM...
       ==> default: Destroying VM and associated drives...
       Vagrant instance <default-ubuntu-1404> destroyed.
       Finished destroying <default-ubuntu-1404> (0m3.56s).
       Finished testing <default-ubuntu-1404> (4m0.89s).

default-ubuntu-1404に対して収束・監査・テストが完了しました。以降、default-ubuntu-1204、default-ubuntu-1004も同様に行われますが省略します。

まとめ

Chef Audit Modeを実際のNodeとtest-kitchenの2つのパターンで試してみました。

NodeにおけるAudit Modeは、Chef社はChef Analyticsとの併用を推奨しています(参考: Chef Audit Mode: CISベンチマーク)。また、Chefによる収束をまったく行わず、単に監査を目的としてのみ使うことも可能です。今回はAudit Mode単体でしたが、今後Chef Analyticsとの併用もテストする予定です。

test-kitchenにおけるAudit Modeは、同じServerspecでCookbookのテストを行うbusser-serverspecの代替となりうるかもしれません。ただ、busser + busser-serverspec + Serverspecの疎結合に比べるとAudit ModeはChef本体とServerspecがあまりにも密に結合しているので、その観点からCookbookのテスト用途としては受け入れ難い方がいるかもしれません。逆に、疎結合と密結合の両方でCookbookのテストをしたいという方もいるでしょう。

いずれにしろServerspecの強力さにより、Chefやインフラにおけるテストやリファクタリングがより便利になったと言えるでしょう。

CL LAB Mail Magazine

CL LABの情報を逃さずチェックしよう!

メールアドレスを登録すると記事が投稿されるとメールで通知します。

メールアドレス: 登録

※登録後メールに記載しているリンクをクリックして認証してください。

Related post