{"id":12984,"date":"2016-03-10T14:11:38","date_gmt":"2016-03-10T05:11:38","guid":{"rendered":"http:\/\/www.creationline.com\/?p=12984"},"modified":"2016-03-10T14:51:21","modified_gmt":"2016-03-10T05:51:21","slug":"hadoop%e3%82%92microsoft-azure%e3%81%a7%e5%8b%95%e3%81%8b%e3%81%97%e3%81%a6%e3%81%bf%e3%82%8b%ef%bc%81%e3%80%8024","status":"publish","type":"post","link":"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/12984","title":{"rendered":"Hadoop\u3092Microsoft Azure\u3067\u52d5\u304b\u3057\u3066\u307f\u308b\uff01\u3000(2\/4)"},"content":{"rendered":"<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-12988\" src=\"\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/hiveR-1.jpg\" alt=\"hive+R\" width=\"377\" height=\"177\" srcset=\"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/hiveR-1.jpg 377w, https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/hiveR-1-360x169.jpg 360w, https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/hiveR-1-202x95.jpg 202w\" sizes=\"auto, (max-width: 377px) 100vw, 377px\" \/><\/p>\n<p>\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3\u6e21\u8fba\u3067\u3059\u3002\u4eca\u56de\u306f<a href=\"\/lab\/12905\" target=\"_blank\" rel=\"noopener\">\u9023\u8f09<\/a>\u306e\u7d9a\u304d\u3067\u3001\u524d\u56de\u4f5c\u6210\u3057\u305fHDInsight\u30af\u30e9\u30b9\u30bf\u30fc\u3092\u4f7f\u3044\u3001Apache Hive\u3068R\u306e\u52d5\u4f5c\u691c\u8a3c\u3092\u884c\u3044\u307e\u3059\u3002\u307e\u305aApache Hive\u3068R\u306e\u6982\u8981\u8aac\u660e\u304b\u3089\u5165\u308a\u3001\u305d\u306e\u5f8c\u5b9f\u969b\u306b\u624b\u3092\u52d5\u304b\u3057\u3066\u3044\u307e\u3059\u3002<\/p>\n<h1>\u7b2c2\u56de\uff1a\u3000Apache Hive\u3068R\u306e\u691c\u8a3c<\/h1>\n<p>&nbsp;<\/p>\n<h2>1.\u00a0<span style=\"text-decoration: underline\">Apache Hive\u306e\u7d39\u4ecb<\/span><\/h2>\n<p>Apache Hive\u306fHadoop\u306b\u8a73\u3057\u3044\u65b9\u306b\u306f\u3059\u3067\u306b\u8aac\u660e\u4e0d\u8981\u3068\u3044\u3046\u3050\u3089\u3044\u30e1\u30b8\u30e3\u30fc\u306aHadoop\u30b3\u30f3\u30dd\u30fc\u30cd\u30f3\u30c8\u3067\u3059\u3002\u6539\u3081\u3066\u89e3\u8aac\u3059\u308b\u3068\u3001Apache Hive\u306f\u4e26\u5217\u51e6\u7406\u57fa\u76e4\u306b\u5bfe\u5fdc\u3059\u308b\u30c7\u30fc\u30bf\u30a6\u30a7\u30a2\u30cf\u30a6\u30b9\u30b7\u30b9\u30c6\u30e0\u3067\u3059\u3002HiveQL\u3068\u3044\u3046SQL\u8a00\u8a9e\u306b\u4f3c\u305f\u30af\u30a8\u30ea\u8a00\u8a9e\u3092\u4f7f\u3044\u3001HDFS\u4e0a\u306e\u30c7\u30fc\u30bf\u306b\u5bfe\u3057\u3066\u3001\u96c6\u8a08\u3084\u5206\u6790\u306a\u3069\u3092\u884c\u3044\u307e\u3059\u3002Hive\u306e\u5229\u70b9\u3068\u3057\u3066\u306f\u5de8\u5927\u306a\u975e\u69cb\u9020\u30c7\u30fc\u30bf\u306b\u69cb\u9020\u3092\u88ab\u305b\u308b\u3053\u3068\u304c\u3067\u304d\u307e\u3059\u3002\u3053\u308c\u3067Java\u3084MapReduce\u306e\u77e5\u8b58\u304c\u306a\u3044\u65b9\u3082\u3001HiveQL\u3092\u4f7f\u3044\u3001SQL\u306b\u8fd1\u3044\u30b9\u30c6\u30fc\u30c8\u30e1\u30f3\u30c8\u3067\u30c7\u30fc\u30bf\u3092\u691c\u7d22\u3059\u308b\u3053\u3068\u304c\u3067\u304d\u307e\u3059\u3002<\/p>\n<h2>2.\u00a0<span style=\"text-decoration: underline\">Apache Hive\u3067Apache log4j\u30d5\u30a1\u30a4\u30eb\u3092\u8abf\u67fb\u3059\u308b<\/span><\/h2>\n<p>\u3067\u306f\u65e9\u901f\u5b9f\u8df5\u306b\u5165\u308a\u307e\u3059\u3002<span style=\"font-weight: 400\">\u6700\u521d\u306bAzure\u30dd\u30fc\u30bf\u30eb\u306b\u30ed\u30b0\u30a4\u30f3\u3057\u3001\u30c0\u30c3\u30b7\u30e5\u30dc\u30fc\u30c9\u306e\u300c\u3059\u3079\u3066\u306e\u30ea\u30bd\u30fc\u30b9\u300d\u304b\u3089\u4f5c\u6210\u3057\u305f\u5c02\u7528\u30af\u30e9\u30b9\u30bf\u30fc\u3092\u9078\u629e\u3057\u307e\u3059\u3002\u753b\u9762\u4e2d\u592e\u306b\u3042\u308b\u30af\u30a4\u30c3\u30af\u30ea\u30f3\u30af\u306e\u4e2d\u304b\u3089\u76e3\u8996\u30c4\u30fc\u30eb\u306e\u4e00\u3064\u3067\u3042\u308bAmbari Views\u3092\u9078\u629e\u3059\u308b\u3068\u30a6\u30a7\u30d6UI\u304c\u958b\u304b\u308c\u307e\u3059\u3002<\/span><\/p>\n<p>\u30dd\u30fc\u30bf\u30eb\u306e\u30c0\u30c3\u30b7\u30e5\u30dc\u30fc\u30c9\u753b\u9762<br \/>\n<img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-12992\" src=\"\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/portal-dashboard.jpg\" alt=\"portal-dashboard\" width=\"434\" height=\"380\" srcset=\"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/portal-dashboard.jpg 434w, https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/portal-dashboard-109x95.jpg 109w\" sizes=\"auto, (max-width: 434px) 100vw, 434px\" \/><\/p>\n<p>WebUI\u304c\u958b\u3044\u305f\u3089\u3001\u30af\u30e9\u30b9\u30bf\u30fc\u4f5c\u6210\u6642\u306b\u767b\u9332\u3057\u305f\u30af\u30e9\u30b9\u30bf\u30fc\u540d\u3068\u30d1\u30b9\u30ef\u30fc\u30c9\u3092\u5165\u529b\u3057\u3001\u30e6\u30fc\u30b6\u8a8d\u8a3c\u3092\u3057\u307e\u3059\u3002<\/p>\n<p><span style=\"font-weight: 400\">Apache Ambari\u306eUI\u306f\u3068\u3066\u3082\u7dba\u9e97\u3067\u3059\u304c\u3001\u52d5\u4f5c\u306b\u3059\u3053\u3057\u4e0d\u53ef\u89e3\u306a\u3068\u3053\u308d\u304c\u3042\u308a\u307e\u3059\u3002\u4e00\u56de\u76ee\u306b\u958b\u3044\u305f\u3068\u304d\u306f\u306a\u305c\u304bAmbari\u304cload\u72b6\u614b\u3067\u8a70\u307e\u308a\u307e\u3059(\u6b63\u3057\u3044\u540d\u524d\u3068\u30d1\u30b9\u3092\u8a18\u5165\u3057\u305f\u306b\u3082\u95a2\u308f\u3089\u305a)\u3002\u307a<\/span><span style=\"font-weight: 400\">\u30fc\u30b8\u66f4\u65b0\u30dc\u30bf\u30f3\u3092\u62bc\u3059\u3068\u7d20\u65e9\u304f\u30ed\u30fc\u30c9\u3055\u308c\u307e\u3059\u3002\u3082\u3046\u4e00\u3064\u6c17\u306b\u306a\u3063\u305f\u3068\u3053\u308d\u306f\u30ed\u30b0\u30a4\u30f3\u3059\u308b\u969b\u306b\u8a8d\u8a3c\u3092\u9593\u9055\u3048\u305f\u3089\u30d6\u30e9\u30a6\u30b6\u30fc\u3092\u4e00\u56de\u9589\u3058\u306a\u3044\u3068\u518d\u8a8d\u8a3c\u3055\u305b\u3066\u304f\u308c\u306a\u304b\u3063\u305f\u6240\u3067\u3059\u3002\u3053\u308c\u306f\u5c11\u3057\u4e0d\u4fbf\u3067\u3059\u3002<\/span><\/p>\n<p>\u8a8d\u8a3c\u3068\u30ed\u30fc\u30c9\u304c\u5b8c\u4e86\u3057\u305f\u3089\u3001\u753b\u9762\u53f3\u4e0a\u306e\u30aa\u30d7\u30b7\u30e7\u30f3\u30dc\u30bf\u30f3\u306e\u4e2d\u304b\u3089Hive View\u3092\u958b\u304d\u307e\u3059\u3002\u3059\u308b\u3068\u4ee5\u4e0b\u306e\u3088\u3046\u306a\u30b9\u30af\u30ea\u30fc\u30f3\u306b\u98db\u3073\u307e\u3059\u3002<\/p>\n<p>Hive View\u753b\u9762<br \/>\n<img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-12993\" src=\"\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/ambari.jpg\" alt=\"ambari\" width=\"541\" height=\"295\" srcset=\"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/ambari.jpg 541w, https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/ambari-174x95.jpg 174w\" sizes=\"auto, (max-width: 541px) 100vw, 541px\" \/><\/p>\n<p>\u4eca\u56de\u4f7f\u7528\u3059\u308b\u30b5\u30f3\u30d7\u30eb\u30d5\u30a1\u30a4\u30eb\u306f\u3001Azure\u5c02\u7528\u30b9\u30c8\u30ec\u30fc\u30b8\u30b7\u30b9\u30c6\u30e0\u3067\u3042\u308bBLOB\u30b3\u30f3\u30c6\u30ca\u306b\u683c\u7d0d\u3055\u308c\u3066\u3044\u308bApache \u306elog4j\u30d5\u30a1\u30a4\u30eb\u3067\u3059\u3002\u3053\u306e\u30d5\u30a1\u30a4\u30eb\u306f\u30b9\u30da\u30fc\u30b9\u3067\u533a\u5207\u3089\u308c\u305f\u3001\u4ee5\u4e0b\u8133\u69d8\u306a\u30c7\u30fc\u30bf\u5f62\u5f0f\u3067\u683c\u7d0d\u3055\u308c\u3066\u3044\u307e\u3059\u3002<\/p>\n<pre>2012-02-03 20:26:41 SampleClass3 [ERROR] verbose detail for id 1527353937<\/pre>\n<p>Query Editor\u3068\u3044\u3046\u5165\u529b\u30dc\u30c3\u30af\u30b9\u5185\u306b\u4ee5\u4e0b\u306e\u30b3\u30de\u30f3\u30c9\u3092\u5165\u308c\u307e\u3059\u3002<\/p>\n<pre>DROP TABLE log4jLogs;\r\nCREATE EXTERNAL TABLE log4jLogs (t1 string, t2 string, t3 string, t4 string, t5 string, t6 string, t7 string)\r\nROW FORMAT DELIMITED FIELDS TERMINATED BY ' '\r\nSTORED AS TEXTFILE LOCATION 'wasb:\/\/\/example\/data\/';\r\nSELECT t4 AS sev, COUNT(*) AS count FROM log4jLogs WHERE t4 = '[ERROR]' AND INPUT__FILE__NAME LIKE '%.log' GROUP BY t4;<\/pre>\n<p>\u3059\u3050\u306b\u7d50\u679c\u304c\u3067\u308b\u306f\u305a\u3067\u3059\u3002\u51e6\u7406\u5f8c\u306e\u7d50\u679c\u306f\u753b\u9762\u4e0b\u306b\u8868\u793a\u3055\u308c\u307e\u3059\u3002<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-13001\" src=\"\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/hive-answer.jpg\" alt=\"hive-answer\" width=\"512\" height=\"382\" srcset=\"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/hive-answer.jpg 512w, https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/hive-answer-127x95.jpg 127w\" sizes=\"auto, (max-width: 512px) 100vw, 512px\" \/><\/p>\n<p>\u3053\u306e\u30d5\u30a1\u30a4\u30eb\u306f3\u3064\u306e\u30a8\u30e9\u30fc\u30ed\u30b0\u304c\u8a18\u9332\u3055\u308c\u3066\u3044\u307e\u3057\u305f\u3002<\/p>\n<p>\u6f14\u7fd2\u7528\u306b<span style=\"font-weight: 400\">GROUP BY \u3092GROUPBY\u306b\u3057\u3066\u30a8\u30e9\u30fc\u753b\u9762\u3092\u30c1\u30a7\u30c3\u30af\u3059\u308b\u3068\u4ee5\u4e0b\u306e\u3088\u3046\u306b\u6559\u3048\u3066\u304f\u308c\u307e\u3059\u3002<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-13021\" src=\"\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/error-message.jpg\" alt=\"error-message\" width=\"618\" height=\"238\" srcset=\"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/error-message.jpg 625w, https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/error-message-225x87.jpg 225w\" sizes=\"auto, (max-width: 618px) 100vw, 618px\" \/><\/p>\n<p>Missing EOF at 'GROUPBY'\u3068\u9069\u5207\u306b\u6559\u3048\u3066\u304f\u308c\u307e\u3059\u3002<\/p>\n<p>\u30af\u30a8\u30ea\u30fc\u3092\u30bb\u30fc\u30d6\u3057\u305f\u3044\u3001\u3082\u3057\u304f\u306f\u30af\u30a8\u30ea\u30fc\u7d50\u679c\u3092\u30bb\u30fc\u30d6\u3057\u305f\u3044\u3068\u3044\u3046\u5834\u5408\u306f\u4ee5\u4e0b\u306e\u756a\u53f7\u304c\u3064\u3044\u305f\u5834\u6240\u3092\u9078\u629e\u3057\u307e\u3059\u3002<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-13022\" src=\"\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/save-options.jpg\" alt=\"save-options\" width=\"569\" height=\"250\" srcset=\"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/save-options.jpg 569w, https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/save-options-216x95.jpg 216w\" sizes=\"auto, (max-width: 569px) 100vw, 569px\" \/><\/p>\n<p><span style=\"font-weight: 400\">1\u306eSave as\u306f\u30af\u30a8\u30ea\u306e\u4fdd\u5b58\u3092\u884c\u3044\u30012\u306eSave results\u306f\u7d50\u679c\u306e\u30c0\u30a6\u30f3\u30ed\u30fc\u30c9\u3092\u884c\u3044\u307e\u3059\u3002\u3053\u308c\u306f\u4fbf\u5229\u3067\u3059\u3002<\/span><\/p>\n<p>Hive View\u3068\u540c\u69d8\u306b\u53f3\u4e0a\u306e\u30aa\u30d7\u30b7\u30e7\u30f3\u306b\u3042\u308bTez View\u3092\u5229\u7528\u3057\u3001\u6709\u52b9\u975e\u5de1\u56de\u30b0\u30e9\u30d5(DAG)\u3092\u89b3\u89a7\u3057\u3066\u307f\u307e\u3059\u3002<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-13026\" src=\"\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/dag-view-1.jpg\" alt=\"dag-view\" width=\"582\" height=\"297\" srcset=\"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/dag-view-1.jpg 594w, https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/dag-view-1-186x95.jpg 186w\" sizes=\"auto, (max-width: 582px) 100vw, 582px\" \/><\/p>\n<p>\u3053\u308c\u3092\u9078\u629e\u3059\u308b\u3068\u6d41\u3057\u305f\u30b8\u30e7\u30d6\u306e\u30b9\u30c6\u30fc\u30b8\u304c\u8a73\u3057\u304f\u793a\u3055\u308c\u3066\u3044\u307e\u3059\u3002\u3082\u3046\u5c11\u3057\u8907\u96d1\u306a\u30b8\u30e7\u30d6\u3092\u6d41\u3059\u3068\u3001\u30d0\u30b0\u8ffd\u8de1\u306b\u5f79\u7acb\u3061\u307e\u3059\u3002<\/p>\n<p>\u4ee5\u4e0a\u304cApache Hive\u3092\u4f7f\u3063\u305f\u7c21\u5358\u306a\u30ed\u30b0\u5206\u6790\u3067\u3057\u305f\u3002\u3053\u306e\u691c\u8a3c\u306fMicrosoft Azure\u306e<a href=\"https:\/\/azure.microsoft.com\/ja-jp\/documentation\/articles\/hdinsight-use-hive\/\">\u516c\u5f0f\u30c1\u30e5\u30fc\u30c8\u30ea\u30a2\u30eb<\/a>\u306b\u57fa\u3065\u3044\u3066\u3044\u307e\u3059\u3002<\/p>\n<p>&nbsp;<\/p>\n<h2>3.\u00a0<span style=\"text-decoration: underline\">R\u3092Hadoop\u74b0\u5883\u3067\u52d5\u304b\u3059<\/span><\/h2>\n<p>\u3053\u308c\u304b\u3089\u30a2\u30af\u30b7\u30e7\u30f3\u30b9\u30af\u30ea\u30d7\u30c8\u3068\u3057\u3066\u8ffd\u52a0\u3057\u305fR\u3092Hadoop\u30b8\u30e7\u30d6\u3068\u3057\u3066\u6d41\u3057\u3066\u3044\u307e\u3059\u3002R\u306f\u30aa\u30fc\u30d7\u30f3\u30bd\u30fc\u30b9\u306e\u7d71\u8a08\u51e6\u7406\u30b7\u30b9\u30c6\u30e0\u3067\u3059\u3002\u30c7\u30fc\u30bf\u30b5\u30a4\u30a8\u30f3\u30c6\u30a3\u30b9\u30c8\u306b\u597d\u307e\u308c\u3001R\u3092\u5229\u7528\u3057\u3066\u3067\u304d\u306a\u3044\u30c7\u30fc\u30bf\u5206\u6790\u306f\u306a\u3044\u3068\u3044\u3046\u307b\u3069\u307e\u3067\u8a00\u308f\u308c\u3066\u3044\u307e\u3059\u304c\u3001\u4e00\u756a\u306e\u8ab2\u984c\u306f\u901f\u5ea6\u3067\u3059\u3002\u3042\u308b\u4e00\u5b9a\u306e\u30c7\u30fc\u30bf\u91cf\u3092\u8d85\u3048\u308b\u3068\u3001\u51e6\u7406\u304c\u975e\u5e38\u306b\u91cd\u304f\u306a\u308a\u307e\u3059\u3002\u305d\u3053\u3067\u3001R\u3092Hadoop\u74b0\u5883\u3067\u4e26\u5217\u51e6\u7406\u3059\u308b\u3053\u3068\u3067\u3053\u306e\u554f\u984c\u3092\u89e3\u6c7a\u3057\u3001\u5f37\u307f\u3067\u3042\u308b\u5e45\u5e83\u3044\u30c7\u30fc\u30bf\u5206\u6790\u3092\u884c\u3044\u307e\u3059\u3002<\/p>\n<p>\u305d\u3053\u3067\u3001\u4e26\u5217\u74b0\u5883\u306b\u5909\u3048\u305fR\u30b8\u30e7\u30d6\u306f\u3069\u308c\u3060\u3051\u9055\u3046\u306e\u304b\u3092\u6bd4\u8f03\u3059\u308b\u691c\u8a3c\u3092\u884c\u3044\u307e\u3059\u3002<br \/>\nR\u306f\u30af\u30e9\u30b9\u30bf\u30fc\u4f5c\u6210\u6642\u306b\u3059\u3067\u306b\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u6e08\u307f\u306a\u306e\u3067\u3001R\u30b3\u30de\u30f3\u30c9\u3092\u547c\u3076\u3060\u3051\u3067\u8d77\u52d5\u3057\u307e\u3059\u3002<\/p>\n<pre>watanabe@hn0-ryuou:~$ R\r\n\r\nR version 3.2.3 (2015-12-10) -- \"Wooden Christmas-Tree\"\r\nCopyright (C) 2015 The R Foundation for Statistical Computing\r\nPlatform: x86_64-pc-linux-gnu (64-bit)<\/pre>\n<p><span style=\"text-decoration: underline\">\u7591\u554f:<\/span> \u3053\u3053\u3067\u3075\u3068\u601d\u3063\u305f\u3053\u3068\u3067\u3059\u304c\u3001\u3082\u3057\u30af\u30e9\u30b9\u30bf\u30fc\u3092\u30b9\u30b1\u30fc\u30eb\u30a2\u30a6\u30c8\u3057\u3001\u30ce\u30fc\u30c9\u6570\u3092\u5897\u3084\u3057\u305f\u3089\u3001\u30a2\u30af\u30b7\u30e7\u30f3\u30b9\u30af\u30ea\u30d7\u30c8\u306a\u3069\u306e\u6a5f\u80fd\u306f\u3059\u3067\u306b\u5404\u30ce\u30fc\u30c9\u306b\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u3055\u308c\u3066\u3044\u308b\u306e\u3067\u3057\u3087\u3046\u304b\uff1f\u3053\u308c\u306f\u30c9\u30ad\u30e5\u30e1\u30f3\u30c8\u8a2d\u5099\u306e\u554f\u984c\u304b\u3082\u3057\u308c\u307e\u305b\u3093\u304c\u3001\u30af\u30e9\u30b9\u30bf\u30fc\u30b9\u30b1\u30fc\u30ea\u30f3\u30b0\u5f8c\u306b\u5185\u90e8\u69cb\u9020\u304c\u3069\u306e\u3088\u3046\u306b\u306a\u308b\u306e\u304b\u306b\u3064\u3044\u3066\u8a00\u53ca\u3055\u308c\u3066\u3044\u308b\u3068\u52a9\u304b\u308a\u307e\u3059\u3002<\/p>\n<p>\u3067\u306f\u65e9\u901fR\u306e\u30b8\u30e7\u30d6\u3092\u6d41\u3057\u307e\u3059\u3002\u521d\u3081\u306bHadoop\u3068\u9023\u643a\u3059\u308b\u305f\u3081\u306b\u5fc5\u8981\u306a\u30e9\u30a4\u30d6\u30e9\u30ea\u3092\u547c\u3073\u3001\u7c21\u5358\u306a\u95a2\u6570\u3092\u5b9f\u884c\u3057\u307e\u3059\u3002<\/p>\n<pre>&gt; library(rmr2)\r\n&gt; ints = to.dfs(1:100)\r\n&gt; calc = mapreduce(input = ints, map = function(k, v) cbind(v, 2*v))<\/pre>\n<p>\u307e\u305a\u30011\u304b\u3089100\u306e\u914d\u5217\u3092\u4f5c\u6210\u3057\u3001HDFS\u4e0a\u306b\u683c\u7d0d\u3057\u307e\u3059\u3002mapReduce\u95a2\u6570\u3092\u4f7f\u3044\u3001\u914d\u5217\u30922\u500d\u306b\u3057\u305f\u6570\u5024\u3068\u4e00\u7dd2\u306b\u30c7\u30fc\u30bf\u30d5\u30ec\u30fc\u30e0\u3092\u4f5c\u6210\u3057\u307e\u3059\u3002Hadoop\u4e0a\u306eR\u30b8\u30e7\u30d6\u306e\u7d50\u679c\u306fHDFS\u4e0a\u304b\u3089\u547c\u3073\u623b\u3055\u306a\u3051\u308c\u3070\u3044\u3051\u307e\u305b\u3093\u306e\u3067\u3001\u4ee5\u4e0b\u306e\u3088\u3046\u306b\u7d50\u679c\u3092\u547c\u3076\u3060\u3051\u3060\u3068\u3001\u683c\u7d0d\u3055\u308c\u3066\u3044\u308b\u30d5\u30a9\u30eb\u30c0\u30fc\u3078\u306e\u30dd\u30a4\u30f3\u30bf\u30fc\u3057\u304b\u8868\u793a\u3055\u308c\u307e\u305b\u3093\u3002<\/p>\n<pre>&gt;print(calc())\r\n[1] \"\/tmp\/fileb10c5b8b52eb\"<\/pre>\n<p>\u7d50\u679c\u3092\u547c\u3073\u51fa\u3059\u65b9\u6cd5\u306ffrom.dfs\u3092\u4f7f\u7528\u3057\u307e\u3059\u3002<\/p>\n<pre>&gt;print(from.dfs(calc()))\r\n$key\r\nNULL\r\n\r\n$val\r\nv\r\n[1,] 1 2\r\n[2,] 2 4\r\n[3,] 3 6\r\n[4,] 4 8\r\n\u2026\u2026\u2026\u2026\r\n[97,] 97 194\r\n[98,] 98 196\r\n[99,] 99 198\r\n[100,] 100 200<\/pre>\n<p>\u5f93\u6765\u306e\u30ed\u30fc\u30ab\u30eb\u306aR\u30b8\u30e7\u30d6\u3068\u6bd4\u8f03\u3057\u3066\u307f\u307e\u3059\u3002<\/p>\n<pre>&gt; v = 1:100\r\n&gt; calc = cbind(v, sapply(v, function(x) x^2))\r\n&gt; print(calc)\r\nv\r\n[1,] 1 1\r\n[2,] 2 4\r\n[3,] 3 9\r\n[4,] 4 16\r\n\u2026\u2026\u2026\u2026\u2026<\/pre>\n<p>Hadoop\u306e\u5834\u5408\u306f\u307e\u305a\u30c7\u30fc\u30bf\u3092HDFS\u4e0a\u306b\u683c\u7d0d\u3057\u306a\u3044\u3068\u3044\u3051\u307e\u305b\u3093\u3002\u3055\u3089\u306b\u30af\u30e9\u30b9\u30bf\u74b0\u5883\u306a\u306e\u3067mapreduce\u95a2\u6570\u3092sapply\u306e\u4ee3\u308f\u308a\u306b\u547c\u3073\u3001\u5404\u30ce\u30fc\u30c9\u9593\u3067\u5171\u6709\u3059\u308b\u5909\u6570\u3092\u6271\u3046\u305f\u3081\u306e\u30ad\u30fc\u30d0\u30ea\u30e5\u2015\u306e\u5f62\u5f0f\u3067\u5b9f\u884c\u3057\u307e\u3059\u3002\u6700\u5f8c\u306b\u51fa\u529b\u3082HDFS\u4e0a\u306b\u3042\u308b\u306e\u3067\u5f93\u6765\u306e\u3084\u308a\u65b9\u3067\u306f\u30dd\u30a4\u30f3\u30bf\u30fc\u3057\u304b\u547c\u3070\u308c\u306a\u3044\u306e\u3067\u3001from.dfs\u3092\u4f7f\u308f\u306a\u3044\u3068\u3044\u3051\u307e\u305b\u3093\u3002<\/p>\n<p>\u7d9a\u3044\u3066\u3001\u3082\u3046\u5c11\u3057\u30d8\u30d3\u30fc\u306a\u51e6\u7406\u3092\u884c\u3044\u307e\u3059\u3002\u4e8c\u9805\u5206\u5e03\u3092\u6c42\u3081\u308b\u95a2\u6570\u3092\u30ed\u30fc\u30ab\u30ebR\u30b8\u30e7\u30d6\u3068HadoopR\u30b8\u30e7\u30d6\u3067\u6bd4\u8f03\u3057\u307e\u3059\u3002<\/p>\n<p>\u30ed\u30fc\u30ab\u30eb\u30b8\u30e7\u30d6<\/p>\n<pre>&gt; groups = rbinom(32, n = 50, prob = 0.4)\r\n&gt; tapply(groups, groups, length)\r\n\r\n7 8 9 10 11 12 13 14 15 16 17 18\r\n1 2 3  1  6  7  9  6  4  6  3  2<\/pre>\n<p>\u65e2\u5b58\u306e\u30e9\u30a4\u30d6\u30e9\u30ea\u304b\u3089rbinom\u95a2\u6570\u3092\u4f7f\u3063\u3066\u3001\u4f8b\u3048\u3070\u30b3\u30a4\u30f3\u304c\u8868\u3068\u306a\u308b\u78ba\u7387\u304c40%\u3042\u3063\u305f\u6642\u306b32\u56de\u4e2d\u4f55\u56de\u8868\u3068\u306a\u3063\u305f\u304b\u3092\u6570\u3048\u307e\u3059\u3002\u3053\u306e\u691c\u67fb\u309250\u56de\u884c\u3044\u3001\u4f55\u56de\u8868\u3068\u306a\u3063\u305f\u306e\u304b\u3092\u7e8f\u3081\u305f\u7b54\u3048\u304c\u4e0a\u8a18\u3068\u306a\u308a\u307e\u3059\u3002<\/p>\n<p>\u7d9a\u3044\u3066Hadoop\u74b0\u5883\u3067\u540c\u3058\u30b8\u30e7\u30d6\u3092\u6d41\u3057\u305f\u3044\u3068\u304d\u306f\u4ee5\u4e0b\u306e\u3088\u3046\u306a\u30b3\u30de\u30f3\u30c9\u306b\u306a\u308a\u307e\u3059\u3002<\/p>\n<pre>&gt; groups = to.dfs(groups)\r\n16\/02\/16 09:02:10 INFO zlib.ZlibFactory: Successfully loaded &amp; initialized native-zlib library\r\n16\/02\/16 09:02:10 INFO compress.CodecPool: Got brand-new compressor [.deflate]\r\n\r\n&gt; from.dfs(mapreduce(input = groups, map = function(., v) keyval(v, 1), reduce = function(k, vv) keyval(k, length(vv))))<\/pre>\n<p>\u307e\u305a\u5148\u307b\u3069\u4f5c\u6210\u3057\u305fgroups\u5909\u6570\u3092HDFS\u4e0a\u306b\u683c\u7d0d\u3057\u307e\u3059\u3002\u666e\u6bb5\u5927\u91cf\u306e\u30c7\u30fc\u30bf\u3092HDFS\u4e0a\u306b\u683c\u7d0d\u3059\u308b\u306e\u306f\u3053\u306e\u65b9\u6cd5\u3067\u306f\u306a\u304f\u3001Flume\u3084Sqoop\u306e\u3088\u3046\u306a\u30b9\u30b1\u30fc\u30e9\u30d6\u30eb\u306b\u30c7\u30fc\u30bf\u79fb\u52d5\u3092\u884c\u3048\u308b\u30b7\u30b9\u30c6\u30e0\u3092\u4f7f\u7528\u3057\u307e\u3059\u3002\u3057\u304b\u3057\u3001\u4eca\u56de\u306e\u5165\u529b\u306fgroups\u3068\u3044\u3046\u5909\u6570\u3092\u4f7f\u7528\u3057\u305f\u30c7\u30fc\u30bf\u306e\u5834\u6240\u3092\u4fdd\u7ba1\u3057\u3066\u3042\u308b\u30d3\u30c3\u30b0\u30c7\u30fc\u30bf\u30aa\u30d6\u30b8\u30a7\u30af\u30c8\u3067\u3059\u306e\u3067\u3001to.dfs\u3092\u5229\u7528\u3057\u307e\u3059\u3002<\/p>\n<p>map\u95a2\u6570\u306f\u5404\u30ce\u30fc\u30c9\u306b\u6563\u3089\u3070\u3063\u305fgroups\u5909\u6570\u306b\u5024\u3059\u308b\u8868\u56de\u6570\u3092\u30ab\u30a6\u30f3\u30c8\u3057\u307e\u3059\u3002reduce\u95a2\u6570\u306f\u30ab\u30a6\u30f3\u30c8\u306e\u6570\u3092\u307e\u3068\u3081\u307e\u3059\u3002<\/p>\n<p>\u4e00\u56de\u76ee\u306e\u5b9f\u884c\u7d50\u679c\u306f\u4ee5\u4e0b\u306e\u3088\u3046\u306a\u51fa\u529b\u3067\u3057\u305f\u3002<\/p>\n<pre>packageJobJar: [] [\/usr\/hdp\/2.2.9.1-1\/hadoop-mapreduce\/hadoop-streaming-2.6.0.2.2.9.1-1.jar] \/tmp\/streamjob2096906629556146005.jar tmpDir=null\r\n16\/02\/16 09:03:07 INFO impl.TimelineClientImpl: Timeline service address: http:\/\/hn0-ryuou.bk4ryjujw10u3g3wen2ztys0zf.lx.internal.cloudapp.net:8188\/ws\/v1\/timeline\/\r\n16\/02\/16 09:03:07 INFO impl.TimelineClientImpl: Timeline service address: http:\/\/hn0-ryuou.bk4ryjujw10u3g3wen2ztys0zf.lx.internal.cloudapp.net:8188\/ws\/v1\/timeline\/\r\n16\/02\/16 09:03:08 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm2\r\n16\/02\/16 09:03:09 INFO mapred.FileInputFormat: Total input paths to process : 1\r\n16\/02\/16 09:03:09 INFO mapreduce.JobSubmitter: number of splits:2\r\n16\/02\/16 09:03:10 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1455523261589_0016\r\n16\/02\/16 09:03:11 INFO impl.YarnClientImpl: Submitted application application_1455523261589_0016\r\n16\/02\/16 09:03:11 INFO mapreduce.Job: The url to track the job: http:\/\/hn1-ryuou.bk4ryjujw10u3g3wen2ztys0zf.lx.internal.cloudapp.net:8088\/proxy\/application_1455523261589_0016\/\r\n16\/02\/16 09:03:11 INFO mapreduce.Job: Running job: job_1455523261589_0016\r\n16\/02\/16 09:03:20 INFO mapreduce.Job: Job job_1455523261589_0016 running in uber mode : false\r\n16\/02\/16 09:03:20 INFO mapreduce.Job: map 0% reduce 0%\r\n16\/02\/16 09:03:25 INFO mapreduce.Job: Task Id : attempt_1455523261589_0016_m_000000_0, Status : FAILED\r\nError: Java heap space\r\n16\/02\/16 09:03:31 INFO mapreduce.Job: Task Id : attempt_1455523261589_0016_m_000000_1, Status : FAILED\r\nError: Java heap space\r\n<\/pre>\n<p>\u30fb\u30fb\u30fb\u898b\u4e8b\u306b\u5931\u6557\u3057\u3066\u3057\u307e\u3044\u307e\u3057\u305f\u3002<\/p>\n<p>\u4e00\u3064\u76ee\u306e\u6bd4\u8f03\u30b8\u30e7\u30d6\u306f\u53b3\u5bc6\u306b\u3044\u3046\u3068map\u51e6\u7406\u3060\u3051\u3092\u884c\u3063\u3066\u3044\u307e\u3057\u305f\u3002\u4e8c\u3064\u76ee\u306e\u30b8\u30e7\u30d6\u306fmap\u51e6\u7406\u3068reduce\u51e6\u7406\u304c\u5165\u3063\u305f\u3082\u306e\u3067\u3042\u308a\u307e\u3059\u3002\u5f93\u3063\u3066\u73fe\u72b6\u306e\u74b0\u5883\u3067\u306fR\u306emapReduce\u30b8\u30e7\u30d6\u306freduce\u3092\u7d44\u307f\u8fbc\u3080\u3068fail\u3059\u308b\u7d50\u679c\u3068\u306a\u308a\u307e\u3057\u305f\u3002<\/p>\n<p>Error\u304cJava heap space\u3068\u3044\u3046\u3053\u3068\u3067\u3001\u30ea\u30bd\u30fc\u30b9\u30fb\u30b8\u30e7\u30d6\u76e3\u8996\u30c4\u30fc\u30eb\u3067\u3042\u308bAmbari\u3092\u4f7f\u7528\u3057\u4fee\u6b63\u3092\u8a66\u307f\u307e\u3057\u305f\u3002<br \/>\n<img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-13053\" src=\"\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/ambari-debug.jpg\" alt=\"ambari-debug\" width=\"606\" height=\"369\" srcset=\"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/ambari-debug.jpg 606w, https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/ambari-debug-156x95.jpg 156w\" sizes=\"auto, (max-width: 606px) 100vw, 606px\" \/><\/p>\n<p><span style=\"font-weight: 400\">Ambari\u753b\u9762\u306eService\u2192MapReduce2\u3092\u9078\u629e\u3059\u308b\u3068\u3001\u521d\u898b\u3068\u306f\u601d\u3048\u306a\u3044\u307b\u3069\u898b\u6163\u308c\u305f\u8a2d\u5b9a\u753b\u9762\u306b\u8fbf\u308a\u3064\u304d\u307e\u3057\u305f\u3002\u5f93\u6765\u8a2d\u5b9a\u3067\u306f1024mb\u306b\u8a2d\u5b9a\u3055\u308c\u3066\u3044\u307e\u3057\u305f\u304c\u3001\u5f90\u3005\u306b\u4e0a\u3052\u3066\u3044\u304d\u3001\u6210\u679c\u304c\u898b\u3089\u308c\u306a\u304b\u3063\u305f\u305f\u3081\u3001\u6700\u7d42\u7684\u306b\u306f\u601d\u3044\u5207\u3063\u30666GB\u307e\u3067\u4e0a\u3052\u307e\u3057\u305f\u3002<\/span><\/p>\n<p>\u3053\u306e\u4e00\u9023\u306e\u64cd\u4f5c\u3067\u611f\u3058\u305f\u3053\u3068\u306fAmbari\u306e\u30ec\u30b3\u30e1\u30f3\u30c9\u8a2d\u5b9a\u306f\u3059\u3054\u304f\u4fbf\u5229\u3067\u3042\u308b\u306b\u3082\u304b\u304b\u308f\u3089\u305a\u3001\u30ea\u30b9\u30bf\u30fc\u30c8\u64cd\u4f5c\u306f\u5c11\u3057\u4e0d\u4fbf\u3067\u3042\u308b\u3053\u3068\u3067\u3059\u3002<\/p>\n<p><span style=\"font-weight: 400\">\u30ec\u30b3\u30e1\u30f3\u30c9\u8a2d\u5b9a\u3068\u3044\u3046\u306e\u306f\u4e00\u3064\u306e\u8a2d\u5b9a\u3092\u5909\u3048\u308b\u3068\u4f9d\u5b58\u6027\u306e\u3042\u308b\u30b3\u30f3\u30d5\u30a3\u30b0\u3092\u77e5\u3089\u305b\u3066\u304f\u308c\u308b\u6a5f\u80fd\u3067\u3059\u3002\u3055\u3089\u306b\u3001\u5909\u66f4\u4e8b\u9805\u306b\u3088\u3063\u3066\u5f71\u97ff\u3092\u53d7\u3051\u305f\u8a2d\u5b9a\u306e\u9069\u5207\u5024\u3092\u63a8\u85a6\u3057\u3066\u304f\u308c\u308b\u306e\u3067\u52a9\u304b\u308a\u307e\u3059\u3002<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-13054\" src=\"\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/recommend-option.jpg\" alt=\"recommend-option\" width=\"606\" height=\"164\" srcset=\"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/recommend-option.jpg 606w, https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/recommend-option-225x61.jpg 225w\" sizes=\"auto, (max-width: 606px) 100vw, 606px\" \/><\/p>\n<p><span style=\"font-weight: 400\">\u518d\u8d77\u52d5\u65b9\u6cd5\u306b\u3064\u3044\u3066\u306f\u5c11\u3057\u6539\u5584\u3067\u304d\u306a\u3044\u304b\u3068\u601d\u3044\u307e\u3057\u305f\u3002\u8a2d\u5b9a\u5909\u66f4\u3092\u53cd\u6620\u3059\u308b\u305f\u3081\u306b\u3001\u30ce\u30fc\u30c9\u3092\u30ea\u30b9\u30bf\u30fc\u30c8\u3059\u308b\u5fc5\u8981\u304c\u3042\u308a\u307e\u3059\u304c\u3001\u5404\u30db\u30b9\u30c8\u6bce\u306e\u30ea\u30b9\u30bf\u30fc\u30c8\u304b\u3001\u5404\u30b5\u30fc\u30d3\u30b9\u306b\u95a2\u4fc2\u3057\u3066\u3044\u308b\u30b3\u30f3\u30dd\u30fc\u30cd\u30f3\u30c8\u6bce\u306b\u30ea\u30b9\u30bf\u30fc\u30c8\u3059\u308b\u65b9\u6cd5\u3057\u304b\u73fe\u72b6\u3067\u306f\u3042\u308a\u307e\u305b\u3093\u3002\u3059\u3067\u306b\u3001\u3042\u308b\u7a0b\u5ea6\u307e\u3068\u3081\u305f\u30ea\u30b9\u30bf\u30fc\u30c8\u65b9\u6cd5\u304c\u5b9f\u88c5\u3055\u308c\u3066\u3044\u307e\u3059\u304c\u3001\u3082\u3046\u4e00\u6b69\u9032\u3093\u3067\u30ea\u30b9\u30bf\u30fc\u30c8\u304c\u5fc5\u8981\u306a\u3082\u306e\u5168\u3066\u3092\u30ea\u30b9\u30bf\u30fc\u30c8\u3059\u308b\u3068\u3044\u3046\u30aa\u30d7\u30b7\u30e7\u30f3\u304c\u3042\u3063\u305f\u3089\u30ce\u30fc\u30c9\u6570\u304c\u304b\u306a\u308a\u591a\u3044\u3068\u304d\u306b\u4fbf\u5229\u3060\u3068\u601d\u3044\u307e\u3059\u3002<del>\u697d\u3092\u3057\u3059\u304e\u304b<\/del><\/span><\/p>\n<p><span style=\"font-weight: 400\">\u30db\u30b9\u30c8\u753b\u9762<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-13061\" src=\"\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/host-screen.jpg\" alt=\"host-screen\" width=\"628\" height=\"247\" srcset=\"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/host-screen.jpg 628w, https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/host-screen-225x88.jpg 225w\" sizes=\"auto, (max-width: 628px) 100vw, 628px\" \/><\/p>\n<p>\u30b5\u30fc\u30d3\u30b9\u753b\u9762<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-13060\" src=\"\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/service-screen.jpg\" alt=\"service-screen\" width=\"645\" height=\"223\" srcset=\"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/service-screen.jpg 645w, https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/service-screen-225x78.jpg 225w\" sizes=\"auto, (max-width: 645px) 100vw, 645px\" \/><\/p>\n<p><span style=\"font-weight: 400\">\u3059\u3079\u3066\u306e\u30ea\u30b9\u30bf\u30fc\u30c8\u3092\u5b8c\u4e86\u3057\u3001\u518d\u5b9f\u884c\u3057\u307e\u3057\u305f\u304c\u307e\u305f<strong>Fail<\/strong>\u3057\u3066\u3044\u307e\u3057\u305f\u3002\u305d\u3053\u3067R\u74b0\u5883\u5185\u3067\u8a2d\u5b9a\u3092\u3044\u3058\u3063\u3066\u307f\u308b\u3053\u3068\u306b\u3057\u307e\u3057\u305f\u3002<\/span><\/p>\n<pre>&gt;help(hadoop.settings)\r\n#\u3092\u8aad\u3080\u3068rmr.options\u3067\u30e1\u30e2\u30ea\u91cf\u3092\u5909\u3048\u3089\u308c\u308b\u3089\u3057\u3044\u3002\r\n#\u73fe\u72b6\u6cc1\u30c1\u30a7\u30c3\u30af\r\nrmr.options(\u201cbackend.parameters\u201d)<\/pre>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-13063\" src=\"\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/r-setting.jpg\" alt=\"r-setting\" width=\"385\" height=\"145\" srcset=\"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/r-setting.jpg 385w, https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/r-setting-225x85.jpg 225w\" sizes=\"auto, (max-width: 385px) 100vw, 385px\" \/><\/p>\n<p>\u4e0a\u56f3\u3092\u898b\u308b\u3068Ambari\u7d4c\u7531\u3067service\u306emapReduce2\u306econfig\u3067memory\u30926GB\u306b\u8abf\u6574\u3057\u307e\u3057\u305f\u304c\u3001\u53cd\u6620\u3055\u308c\u3066\u3044\u306a\u3044\u6a21\u69d8\u3067\u3059\u3002<\/p>\n<p>\u305d\u3053\u3067R\u74b0\u5883\u5185\u3067Java\u30e1\u30e2\u30ea\u8a2d\u5b9a\u3092\u884c\u3044\u307e\u3059\u3002<\/p>\n<pre>#\u30d1\u30e9\u30e1\u30fc\u30bf\u5909\u6570\u4f5c\u6210\r\n&gt;bp = rmr.options(\u201cbackend.parameters\u201d)\r\n\r\n#\u30d1\u30e9\u30e1\u30fc\u30bf\u5909\u6570\u5909\u66f4\r\n&gt;bp$hadoop[1] = \u201cmapreduce.map.java.opts=-Xmx4092M\u201d\r\n&gt;bp$hadoop[2] = \u201cmapreduce.reduce.java.opts=-Xmx4092M\u201d\r\n\r\n#\r\n&gt;rmr.options(backend.parameters=bp)<\/pre>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-13064\" src=\"\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/new-r-setting.jpg\" alt=\"new-r-setting\" width=\"341\" height=\"178\" srcset=\"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/new-r-setting.jpg 341w, https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/new-r-setting-182x95.jpg 182w\" sizes=\"auto, (max-width: 341px) 100vw, 341px\" \/><\/p>\n<p>\u8a2d\u5b9a\u304c\u5909\u66f4\u3055\u308c\u305f\u3068\u3053\u308d\u3067\u518d\u6311\u6226\u3057\u307e\u3059\u3002<\/p>\n<pre>&gt; groups = to.dfs(rbinom(32, n = 50, prob = 0.4))\r\n16\/02\/17 09:02:10 INFO zlib.ZlibFactory: Successfully loaded &amp; initialized native-zlib library\r\n16\/02\/17 09:02:10 INFO compress.CodecPool: Got brand-new compressor [.deflate]\r\n\r\n&gt; from.dfs(mapreduce(input = groups, map = function(., v) keyval(v, 1), reduce = function(k, vv) keyval(k, length(vv))))\r\npackageJobJar: [] [\/usr\/hdp\/2.2.9.1-1\/hadoop-mapreduce\/hadoop-streaming-2.6.0.2.2.9.1-1.jar] \/tmp\/streamjob9130136134571085855.jar tmpDir=null\r\n16\/02\/17 09:04:37 INFO impl.TimelineClientImpl: Timeline service address: http:\/\/hn0-ryuou.bk4ryjujw10u3g3wen2ztys0zf.lx.internal.cloudapp.net:8188\/ws\/v1\/timeline\/\r\n16\/02\/17 09:04:38 INFO impl.TimelineClientImpl: Timeline service address: http:\/\/hn0-ryuou.bk4ryjujw10u3g3wen2ztys0zf.lx.internal.cloudapp.net:8188\/ws\/v1\/timeline\/\r\n16\/02\/17 09:04:38 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm2\r\n16\/02\/17 09:04:40 INFO mapred.FileInputFormat: Total input paths to process : 1\r\n16\/02\/17 09:04:40 INFO mapreduce.JobSubmitter: number of splits:2\r\n16\/02\/17 09:04:41 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1455692794185_0003\r\n16\/02\/17 09:04:41 INFO impl.YarnClientImpl: Submitted application application_1455692794185_0003\r\n16\/02\/17 09:04:41 INFO mapreduce.Job: The url to track the job: http:\/\/hn1-ryuou.bk4ryjujw10u3g3wen2ztys0zf.lx.internal.cloudapp.net:8088\/proxy\/application_1455692794185_0003\/\r\n16\/02\/17 09:04:41 INFO mapreduce.Job: Running job: job_1455692794185_0003\r\n16\/02\/17 09:04:51 INFO mapreduce.Job: Job job_1455692794185_0003 running in uber mode : false\r\n16\/02\/17 09:04:51 INFO mapreduce.Job: map 0% reduce 0%\r\n16\/02\/17 09:04:59 INFO mapreduce.Job: map 100% reduce 0%\r\n16\/02\/17 09:05:06 INFO mapreduce.Job: map 100% reduce 100%\r\n16\/02\/17 09:05:08 INFO mapreduce.Job: Job job_1455692794185_0003 completed successfully\r\n16\/02\/17 09:05:08 INFO mapreduce.Job: Counters: 50\r\n\u2026\u2026\u2026\u2026\u2026\u2026\u2026\u2026\u2026\u2026\u2026\u2026\u2026\u2026\u2026\u2026\u2026\u2026\u2026\u2026\u2026\u2026\u2026\u2026\r\n16\/02\/17 09:05:08 INFO streaming.StreamJob: Output directory: \/tmp\/fileb10c6b3a1afb\r\n16\/02\/17 09:05:16 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval = 360 minutes, Emptier interval = 0 minutes.\r\nMoved: 'wasb:\/\/ryuou@akibaline.blob.core.windows.net\/tmp\/fileb10c5b8b52eb' to trash at: wasb:\/\/ryuou@akibaline.blob.core.windows.net\/user\/watanabe\/.Trash\/Current\r\n$key\r\n[1] 5 7 8 9 10 11 12 13 14 15 16 17 18 19 20\r\n\r\n$val\r\n[1] 1 1 1 4 4 4 7 7 7 4 4 2 2 1 1<\/pre>\n<p>\u898b\u4e8b\u306b\u6d41\u308c\u3066\u304f\u308c\u307e\u3057\u305f\uff01<\/p>\n<p>\u3053\u306eR\u306e\u691c\u8a3c\u306e<a href=\"https:\/\/azure.microsoft.com\/ja-jp\/documentation\/articles\/hdinsight-hadoop-r-scripts-linux\/\">\u516c\u5f0f\u7248<\/a>\u306f\u3053\u3061\u3089\u3067\u3059\u3002<\/p>\n<h2>\u3053\u3053\u307e\u3067\u306e\u611f\u60f3<\/h2>\n<p>R\u306emapreduce\u30b8\u30e7\u30d6\u306f\u72ec\u81ea\u306e\u8a2d\u5b9a\u30d5\u30a1\u30a4\u30eb\u3092\u53c2\u7167\u3057\u3066\u3044\u308b\u3068\u601d\u308f\u308c\u307e\u3059\u3002Ambari\u3067\u8a2d\u5b9a\u3092\u5909\u3048\u3089\u308c\u308b\u65b9\u6cd5\u304c\u3042\u308b\u306e\u304b\u3001\u307e\u305f\u306f\u8a2d\u5b9a\u3092\u9023\u643a\u3055\u305b\u308b\u3053\u3068\u304c\u3067\u304d\u308b\u306e\u304b\u304c\u4e0d\u900f\u660e\u3067\u3059\u3002\u3068\u3082\u3042\u308cAmabri\u4e0a\u3067config\u3092\u76f4\u611f\u7684\u306b\u5909\u3048\u3089\u308c\u308b\u306e\u3067\u3001\u5909\u66f4\u304c\u8ffd\u8de1\u3067\u304d\u308b\u306e\u306f\u3042\u308a\u304c\u305f\u3044\u3067\u3059\u3002<\/p>\n<p>\u30d3\u30c3\u30b0\u30c7\u30fc\u30bf\u3092R\u3067\u3082\u51e6\u7406\u3067\u304d\u308b\u3088\u3046\u306bHadoop\u3092\u7d4c\u7531\u3059\u308b\u3068\u30ed\u30b8\u30c3\u30af\u306f\u540c\u3058\u3060\u304c\u74b0\u5883\u306b\u5408\u308f\u305b\u305f\u30d7\u30ed\u30b0\u30e9\u30df\u30f3\u30b0\u304c\u5fc5\u8981\u306b\u306a\u308a\u307e\u3059\u3002<\/p>\n<p>\u6b21\u56de\u306fGiraph\u3092Hadoop\u3067\u52d5\u304b\u3057\u3066\u307f\u307e\u3059\uff01<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3\u6e21\u8fba\u3067\u3059\u3002\u4eca\u56de\u306f\u9023\u8f09\u306e\u7d9a\u304d\u3067\u3001\u524d\u56de\u4f5c\u6210\u3057\u305fHDInsight\u30af\u30e9\u30b9\u30bf\u30fc\u3092\u4f7f\u3044\u3001Apache Hive\u3068R\u306e\u52d5\u4f5c\u691c\u8a3c\u3092\u884c\u3044\u307e\u3059\u3002\u307e\u305aApache Hive\u3068R\u306e\u6982\u8981\u8aac\u660e\u304b\u3089\u5165\u308a\u3001\u305d\u306e\u5f8c\u5b9f\u969b\u306b\u624b\u3092\u52d5\u304b\u3057\u3066 [&#8230;]<\/p>\n","protected":false},"author":1,"featured_media":12988,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"content-type":"","footnotes":""},"categories":[58,16],"tags":[66,65],"class_list":["post-12984","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-azure","category-author","tag-hdinsight","tag-microsoft-azure"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.4 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Hadoop\u3092Microsoft Azure\u3067\u52d5\u304b\u3057\u3066\u307f\u308b\uff01\u3000(2\/4) - Tech Blog\uff5c\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3<\/title>\n<meta name=\"description\" content=\"Azure, \u8457\u8005\uff08Author\uff09 |\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3\u6e21\u8fba\u3067\u3059\u3002\u4eca\u56de\u306f\u9023\u8f09\u306e\u7d9a\u304d\u3067\u3001\u524d\u56de\u4f5c\u6210\u3057\u305fHDInsight\u30af\u30e9\u30b9\u30bf\u30fc\u3092\u4f7f\u3044\u3001Apache\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/12984\" \/>\n<meta property=\"og:locale\" content=\"ja_JP\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Hadoop\u3092Microsoft Azure\u3067\u52d5\u304b\u3057\u3066\u307f\u308b\uff01\u3000(2\/4) - Tech Blog\uff5c\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3\" \/>\n<meta property=\"og:description\" content=\"Azure, \u8457\u8005\uff08Author\uff09 |\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3\u6e21\u8fba\u3067\u3059\u3002\u4eca\u56de\u306f\u9023\u8f09\u306e\u7d9a\u304d\u3067\u3001\u524d\u56de\u4f5c\u6210\u3057\u305fHDInsight\u30af\u30e9\u30b9\u30bf\u30fc\u3092\u4f7f\u3044\u3001Apache\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/12984\" \/>\n<meta property=\"og:site_name\" content=\"Tech Blog\uff5c\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/creationline\" \/>\n<meta property=\"article:published_time\" content=\"2016-03-10T05:11:38+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2016-03-10T05:51:21+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/hiveR-1.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"377\" \/>\n\t<meta property=\"og:image:height\" content=\"177\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"admin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@creationline\" \/>\n<meta name=\"twitter:site\" content=\"@creationline\" \/>\n<meta name=\"twitter:label1\" content=\"\u57f7\u7b46\u8005\" \/>\n\t<meta name=\"twitter:data1\" content=\"admin\" \/>\n\t<meta name=\"twitter:label2\" content=\"\u63a8\u5b9a\u8aad\u307f\u53d6\u308a\u6642\u9593\" \/>\n\t<meta name=\"twitter:data2\" content=\"4\u5206\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cloudnative\\\/azure\\\/12984#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cloudnative\\\/azure\\\/12984\"},\"author\":{\"name\":\"admin\",\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/#\\\/schema\\\/person\\\/7d923d1c017568a1a5e66d7bb1c8764a\"},\"headline\":\"Hadoop\u3092Microsoft Azure\u3067\u52d5\u304b\u3057\u3066\u307f\u308b\uff01\u3000(2\\\/4)\",\"datePublished\":\"2016-03-10T05:11:38+00:00\",\"dateModified\":\"2016-03-10T05:51:21+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cloudnative\\\/azure\\\/12984\"},\"wordCount\":159,\"image\":{\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cloudnative\\\/azure\\\/12984#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cms_x3GWkuX\\\/wp-content\\\/uploads\\\/2016\\\/03\\\/hiveR-1.jpg\",\"keywords\":[\"HDInsight\",\"Microsoft Azure\"],\"articleSection\":[\"Azure\",\"\u8457\u8005\uff08Author\uff09\"],\"inLanguage\":\"ja\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cloudnative\\\/azure\\\/12984\",\"url\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cloudnative\\\/azure\\\/12984\",\"name\":\"Hadoop\u3092Microsoft Azure\u3067\u52d5\u304b\u3057\u3066\u307f\u308b\uff01\u3000(2\\\/4) - Tech Blog\uff5c\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cloudnative\\\/azure\\\/12984#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cloudnative\\\/azure\\\/12984#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cms_x3GWkuX\\\/wp-content\\\/uploads\\\/2016\\\/03\\\/hiveR-1.jpg\",\"datePublished\":\"2016-03-10T05:11:38+00:00\",\"dateModified\":\"2016-03-10T05:51:21+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/#\\\/schema\\\/person\\\/7d923d1c017568a1a5e66d7bb1c8764a\"},\"description\":\"Azure, \u8457\u8005\uff08Author\uff09 |\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3\u6e21\u8fba\u3067\u3059\u3002\u4eca\u56de\u306f\u9023\u8f09\u306e\u7d9a\u304d\u3067\u3001\u524d\u56de\u4f5c\u6210\u3057\u305fHDInsight\u30af\u30e9\u30b9\u30bf\u30fc\u3092\u4f7f\u3044\u3001Apache\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cloudnative\\\/azure\\\/12984#breadcrumb\"},\"inLanguage\":\"ja\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cloudnative\\\/azure\\\/12984\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"ja\",\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cloudnative\\\/azure\\\/12984#primaryimage\",\"url\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cms_x3GWkuX\\\/wp-content\\\/uploads\\\/2016\\\/03\\\/hiveR-1.jpg\",\"contentUrl\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cms_x3GWkuX\\\/wp-content\\\/uploads\\\/2016\\\/03\\\/hiveR-1.jpg\",\"width\":377,\"height\":177},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cloudnative\\\/azure\\\/12984#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"HOME\",\"item\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"\u30af\u30e9\u30a6\u30c9\u30cd\u30a4\u30c6\u30a3\u30d6\",\"item\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cloudnative\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Azure\",\"item\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cloudnative\\\/azure\"},{\"@type\":\"ListItem\",\"position\":4,\"name\":\"Hadoop\u3092Microsoft Azure\u3067\u52d5\u304b\u3057\u3066\u307f\u308b\uff01\u3000(2\\\/4)\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/#website\",\"url\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/\",\"name\":\"Tech Blog\uff5c\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3\",\"description\":\"\u30a2\u30b8\u30e3\u30a4\u30eb\uff06DevOps\u3001\u30af\u30e9\u30a6\u30c9\u30cd\u30a4\u30c6\u30a3\u30d6\u3001AI\uff06LLM\u306e\u5148\u7aef\u6280\u8853\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"ja\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/#\\\/schema\\\/person\\\/7d923d1c017568a1a5e66d7bb1c8764a\",\"name\":\"admin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"ja\",\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cms_x3GWkuX\\\/wp-content\\\/uploads\\\/2021\\\/12\\\/avatar.png\",\"url\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cms_x3GWkuX\\\/wp-content\\\/uploads\\\/2021\\\/12\\\/avatar.png\",\"contentUrl\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cms_x3GWkuX\\\/wp-content\\\/uploads\\\/2021\\\/12\\\/avatar.png\",\"caption\":\"admin\"},\"url\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/author\\\/admin\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Hadoop\u3092Microsoft Azure\u3067\u52d5\u304b\u3057\u3066\u307f\u308b\uff01\u3000(2\/4) - Tech Blog\uff5c\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3","description":"Azure, \u8457\u8005\uff08Author\uff09 |\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3\u6e21\u8fba\u3067\u3059\u3002\u4eca\u56de\u306f\u9023\u8f09\u306e\u7d9a\u304d\u3067\u3001\u524d\u56de\u4f5c\u6210\u3057\u305fHDInsight\u30af\u30e9\u30b9\u30bf\u30fc\u3092\u4f7f\u3044\u3001Apache","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/12984","og_locale":"ja_JP","og_type":"article","og_title":"Hadoop\u3092Microsoft Azure\u3067\u52d5\u304b\u3057\u3066\u307f\u308b\uff01\u3000(2\/4) - Tech Blog\uff5c\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3","og_description":"Azure, \u8457\u8005\uff08Author\uff09 |\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3\u6e21\u8fba\u3067\u3059\u3002\u4eca\u56de\u306f\u9023\u8f09\u306e\u7d9a\u304d\u3067\u3001\u524d\u56de\u4f5c\u6210\u3057\u305fHDInsight\u30af\u30e9\u30b9\u30bf\u30fc\u3092\u4f7f\u3044\u3001Apache","og_url":"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/12984","og_site_name":"Tech Blog\uff5c\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3","article_publisher":"https:\/\/www.facebook.com\/creationline","article_published_time":"2016-03-10T05:11:38+00:00","article_modified_time":"2016-03-10T05:51:21+00:00","og_image":[{"width":377,"height":177,"url":"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/hiveR-1.jpg","type":"image\/jpeg"}],"author":"admin","twitter_card":"summary_large_image","twitter_creator":"@creationline","twitter_site":"@creationline","twitter_misc":{"\u57f7\u7b46\u8005":"admin","\u63a8\u5b9a\u8aad\u307f\u53d6\u308a\u6642\u9593":"4\u5206"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/12984#article","isPartOf":{"@id":"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/12984"},"author":{"name":"admin","@id":"https:\/\/www.creationline.com\/tech-blog\/#\/schema\/person\/7d923d1c017568a1a5e66d7bb1c8764a"},"headline":"Hadoop\u3092Microsoft Azure\u3067\u52d5\u304b\u3057\u3066\u307f\u308b\uff01\u3000(2\/4)","datePublished":"2016-03-10T05:11:38+00:00","dateModified":"2016-03-10T05:51:21+00:00","mainEntityOfPage":{"@id":"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/12984"},"wordCount":159,"image":{"@id":"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/12984#primaryimage"},"thumbnailUrl":"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/hiveR-1.jpg","keywords":["HDInsight","Microsoft Azure"],"articleSection":["Azure","\u8457\u8005\uff08Author\uff09"],"inLanguage":"ja"},{"@type":"WebPage","@id":"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/12984","url":"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/12984","name":"Hadoop\u3092Microsoft Azure\u3067\u52d5\u304b\u3057\u3066\u307f\u308b\uff01\u3000(2\/4) - Tech Blog\uff5c\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3","isPartOf":{"@id":"https:\/\/www.creationline.com\/tech-blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/12984#primaryimage"},"image":{"@id":"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/12984#primaryimage"},"thumbnailUrl":"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/hiveR-1.jpg","datePublished":"2016-03-10T05:11:38+00:00","dateModified":"2016-03-10T05:51:21+00:00","author":{"@id":"https:\/\/www.creationline.com\/tech-blog\/#\/schema\/person\/7d923d1c017568a1a5e66d7bb1c8764a"},"description":"Azure, \u8457\u8005\uff08Author\uff09 |\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3\u6e21\u8fba\u3067\u3059\u3002\u4eca\u56de\u306f\u9023\u8f09\u306e\u7d9a\u304d\u3067\u3001\u524d\u56de\u4f5c\u6210\u3057\u305fHDInsight\u30af\u30e9\u30b9\u30bf\u30fc\u3092\u4f7f\u3044\u3001Apache","breadcrumb":{"@id":"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/12984#breadcrumb"},"inLanguage":"ja","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/12984"]}]},{"@type":"ImageObject","inLanguage":"ja","@id":"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/12984#primaryimage","url":"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/hiveR-1.jpg","contentUrl":"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/hiveR-1.jpg","width":377,"height":177},{"@type":"BreadcrumbList","@id":"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/12984#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"HOME","item":"https:\/\/www.creationline.com\/tech-blog"},{"@type":"ListItem","position":2,"name":"\u30af\u30e9\u30a6\u30c9\u30cd\u30a4\u30c6\u30a3\u30d6","item":"https:\/\/www.creationline.com\/tech-blog\/cloudnative"},{"@type":"ListItem","position":3,"name":"Azure","item":"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure"},{"@type":"ListItem","position":4,"name":"Hadoop\u3092Microsoft Azure\u3067\u52d5\u304b\u3057\u3066\u307f\u308b\uff01\u3000(2\/4)"}]},{"@type":"WebSite","@id":"https:\/\/www.creationline.com\/tech-blog\/#website","url":"https:\/\/www.creationline.com\/tech-blog\/","name":"Tech Blog\uff5c\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3","description":"\u30a2\u30b8\u30e3\u30a4\u30eb\uff06DevOps\u3001\u30af\u30e9\u30a6\u30c9\u30cd\u30a4\u30c6\u30a3\u30d6\u3001AI\uff06LLM\u306e\u5148\u7aef\u6280\u8853","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.creationline.com\/tech-blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"ja"},{"@type":"Person","@id":"https:\/\/www.creationline.com\/tech-blog\/#\/schema\/person\/7d923d1c017568a1a5e66d7bb1c8764a","name":"admin","image":{"@type":"ImageObject","inLanguage":"ja","@id":"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2021\/12\/avatar.png","url":"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2021\/12\/avatar.png","contentUrl":"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2021\/12\/avatar.png","caption":"admin"},"url":"https:\/\/www.creationline.com\/tech-blog\/author\/admin"}]}},"_links":{"self":[{"href":"https:\/\/www.creationline.com\/tech-blog\/wp-json\/wp\/v2\/posts\/12984","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.creationline.com\/tech-blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.creationline.com\/tech-blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.creationline.com\/tech-blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.creationline.com\/tech-blog\/wp-json\/wp\/v2\/comments?post=12984"}],"version-history":[{"count":47,"href":"https:\/\/www.creationline.com\/tech-blog\/wp-json\/wp\/v2\/posts\/12984\/revisions"}],"predecessor-version":[{"id":13189,"href":"https:\/\/www.creationline.com\/tech-blog\/wp-json\/wp\/v2\/posts\/12984\/revisions\/13189"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.creationline.com\/tech-blog\/wp-json\/wp\/v2\/media\/12988"}],"wp:attachment":[{"href":"https:\/\/www.creationline.com\/tech-blog\/wp-json\/wp\/v2\/media?parent=12984"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.creationline.com\/tech-blog\/wp-json\/wp\/v2\/categories?post=12984"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.creationline.com\/tech-blog\/wp-json\/wp\/v2\/tags?post=12984"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}