{"id":13093,"date":"2016-03-16T13:53:46","date_gmt":"2016-03-16T04:53:46","guid":{"rendered":"http:\/\/www.creationline.com\/?p=13093"},"modified":"2016-11-05T14:18:22","modified_gmt":"2016-11-05T05:18:22","slug":"hadoop%e3%82%92microsoft-azure%e4%b8%8a%e3%81%a7%e5%8b%95%e3%81%8b%e3%81%97%e3%81%a6%e3%81%bf%e3%82%8b%ef%bc%81%e3%80%8034","status":"publish","type":"post","link":"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/13093","title":{"rendered":"Hadoop\u3092Microsoft Azure\u4e0a\u3067\u52d5\u304b\u3057\u3066\u307f\u308b\uff01\u3000(3\/4)"},"content":{"rendered":"

\"giraph-azur\"<\/p>\n

\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3\u6e21\u8fba\u3067\u3059\u3002\u4eca\u56de\u306f\u9023\u8f09\u306e\u7b2c\u4e09\u56de\u3067\u3001Hadoop\u30a8\u30b3\u30b7\u30b9\u30c6\u30e0\u306e\u4e00\u3064\u3067\u3042\u308b\u4e26\u5217\u30b0\u30e9\u30d5\u51e6\u7406\u57fa\u76e4\u300cGiraph\u300d\u3092Azure\u4e0a\u3067\u52d5\u304b\u3057\u3066\u307f\u307e\u3059\u3002\u30c7\u30fc\u30bf\u304c\u30cf\u30c1\u306e\u5de3\u306e\u3088\u3046\u306b\u96c6\u307e\u308bApache Hive<\/a>\u306e\u6b21\u306f\u30ad\u30ea\u30f3\u306e\u9996\u306e\u69d8\u306b\u30b9\u30b1\u30fc\u30eb\u30a2\u30a6\u30c8\u3067\u304d\u308b\u30b0\u30e9\u30d5\u57fa\u76e4Giraph\u3067\u3059\u3002<\/p>\n

\u7b2c\u4e09\u56de\uff1a\u3000Giraph\u3067\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u6700\u77ed\u8ddd\u96e2\u3092\u63a2\u7d22\u3059\u308b<\/h1>\n

 <\/p>\n

\uff11\uff0eApache Giraph\u306e\u7d39\u4ecb<\/span><\/h2>\n

Apache Giraph\u306fAzure HDInsight\u306b\u5bfe\u5fdc\u3057\u3066\u3044\u3066\u3001\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u5206\u6790\u306a\u3069\u3067\u5fdc\u7528\u3055\u308c\u308b\u30b0\u30e9\u30d5\u51e6\u7406\u3092\u30b9\u30b1\u30fc\u30e9\u30d6\u30eb\u306a\u74b0\u5883\u3067\u884c\u3046\u305f\u3081\u306e\u30b7\u30b9\u30c6\u30e0\u3067\u3059\u3002\u73fe\u306bFacebook\u793e\u304c\u81ea\u793e\u306e\u5de8\u5927\u306a\u30bd\u30fc\u30b7\u30e3\u30eb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u306e\u7ba1\u7406\u3001\u5206\u6790\u306b\u6d3b\u7528<\/a>\u3057\u3066\u3044\u308b\u3053\u3068\u3067\u6709\u540d\u3067\u3059\u3002Giraph\u306fJava\u3067\u958b\u767a\u3055\u308c\u3066\u3044\u3066\u3001\u5229\u70b9\u306f\u3084\u306f\u308aHadoop\u3068\u9023\u643a\u3057\u3066\u30b0\u30e9\u30d5\u8a08\u7b97\u3092Hadoop\u30b8\u30e7\u30d6\u3068\u3057\u3066\u5927\u91cf\u306e\u30c7\u30fc\u30bf\u306b\u5bfe\u3057\u3066\u6d41\u3059\u3053\u3068\u304c\u3067\u304d\u308b\u3068\u3044\u3046\u3053\u3068\u3067\u3059\u3002<\/p>\n

\u30bd\u30fc\u30b7\u30e3\u30eb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u5206\u6790\u4ee5\u5916\u306b\u3082\u3001\u30b0\u30e9\u30d5\u51e6\u7406\u306f\u30a6\u30a7\u30d6\u306e\u30da\u30fc\u30b8\u30e9\u30f3\u30ad\u30f3\u30b0\u3084\u30b3\u30f3\u30d4\u30e5\u30fc\u30bf\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u9593\u306e\u6700\u77ed\u8ddd\u96e2\u306a\u3069\u3092\u8abf\u67fb\u3059\u308b\u3053\u3068\u306b\u4fbf\u5229\u3067\u3059\u3002CL Lab\u3067\u306f\u666e\u6bb5\u3042\u307e\u308a\u898b\u3089\u308c\u306a\u3044\u30e6\u30cb\u30fc\u30af\u306a\u5fdc\u7528<\/a>\u3082\u7d39\u4ecb\u3057\u3066\u3044\u307e\u3059\u3002\u3053\u3061\u3089\u3082\u305c\u3072\u3054\u53c2\u7167\u304f\u3060\u3055\u3044\u3002<\/p>\n

Apache Giraph\u306e\u30c7\u30fc\u30bf\u30d5\u30ed\u30fc<\/p>\n

\"giraph-flow\"<\/p>\n

\u4e0a\u8a18\u304cGiraph\u306b\u304a\u3051\u308b\u4e00\u822c\u7684\u306a\u30c7\u30fc\u30bf\u30d5\u30ed\u30fc\u30c1\u30e3\u30fc\u30c8\u3067\u3059\u3002\u307e\u305a\u30de\u30b9\u30bf\u30fc\u30ce\u30fc\u30c9\u304b\u3089\u6307\u5b9a\u3055\u308c\u305f\u30d5\u30a9\u30fc\u30de\u30c3\u30c8\u306e\u5165\u529b\u304c\u30ef\u30fc\u30ab\u30fc\u30ce\u30fc\u30c9\u306b\u30ed\u30fc\u30c9\u3055\u308c\u307e\u3059\u3002\u6b21\u306b\u30e1\u30e2\u30ea\u5185\u306b\u7f6e\u304b\u308c\u305f\u30b0\u30e9\u30d5\u306b\u5bfe\u3057\u3066\u5404\u30ef\u30fc\u30ab\u30fc\u30ce\u30fc\u30c9\u6bce\u306b\u7e70\u308a\u8fd4\u3057\u306e\u51e6\u7406\u304c\u884c\u308f\u308c\u307e\u3059\u3002\u3053\u306e\u9593\u30ef\u30fc\u30ab\u30fc\u30ce\u30fc\u30c9\u9593\u3068\u30ef\u30fc\u30ab\u30fc\u30ce\u30fc\u30c9\u304b\u3089\u30de\u30b9\u30bf\u30fc\u3078\u306e\u901a\u4fe1\u306f\u5e38\u306b\u884c\u308f\u308c\u3066\u3044\u308b\u72b6\u614b\u3067\u3059\u3002\u6700\u5f8c\u306b\u5206\u5272\u3055\u308c\u305f\u30b0\u30e9\u30d5\u304c\u6307\u5b9a\u3055\u308c\u305f\u51fa\u529b\u30d5\u30a1\u30a4\u30eb\u3068\u3057\u3066\u4fdd\u7ba1\u3055\u308c\u307e\u3059\u3002<\/p>\n

2. \u5b9f\u8df5<\/span><\/h2>\n

\u3067\u306f\u65e9\u901f\u5b9f\u8df5\u306b\u5165\u308a\u307e\u3059\u3002\u4ee5\u4e0b\u306e\u56f3\u306e\u3088\u3046\u306a\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u56f3\u3092\u4f5c\u6210\u3057\u3001\u30ce\u30fc\u30c9ID1\u304b\u3089\u4ed6\u306e\u30ce\u30fc\u30c9\u3078\u306e\u6700\u77ed\u8ddd\u96e2\u3092\u63a2\u7d22\u3057\u307e\u3059\u3002<\/p>\n

\"sample-network\"<\/p>\n

\u5404\u9802\u70b9(\u30ce\u30fc\u30c9)\u306b\u306fID\u304c\u632f\u3089\u308c\u3066\u304a\u308a\u3001\u30ce\u30fc\u30c9\u9593\u306e\u8fba(\u30a8\u30c3\u30b8)\u306b\u306f\u8ddd\u96e2\u306b\u306a\u308b\u8981\u7d20\u304c\u542b\u307e\u308c\u3066\u3044\u307e\u3059\u3002\u4e0a\u56f3\u306e\u30c7\u30fc\u30bf\u3092\u4f5c\u6210\u3059\u308b\u305f\u3081\u306b\u3001vi \u30a8\u30c7\u30a3\u30bf\u3092\u958b\u3044\u3066\u4ee5\u4e0b\u3092\u66f8\u304d\u8fbc\u307f\u307e\u3059\u3002<\/span><\/p>\n

[0,0,[[1,1],[3,3]]]
\n[1,0,[[0,1],[2,2],[3,1]]]
\n[2,0,[[1,2],[4,4]]]
\n[3,0,[[0,3],[1,1],[4,4]]]
\n[4,0,[[3,4],[2,4]]]<\/code><\/p>\n

\u3053\u306e\u30c7\u30fc\u30bf\u306f[source_id, source_value,[ [destination_id, edge_value],[..., ...] ] ]\u3068\u3044\u3046\u5f62\u5f0f\u306b\u306a\u3063\u3066\u3044\u307e\u3059\u3002<\/span>\u8981\u3059\u308b\u306b[\u63a5\u7d9a\u5143ID, \u63a5\u7d9a\u5143\u5024, [ [ \u63a5\u7d9a\u5148ID, \u8fba\u5024], [..., ...] ] ] \u3067\u3042\u308a\u307e\u3059\u3002<\/span><\/p>\n

\u6bd4\u8f03\u5bfe\u8c61\u3068\u3057\u3066Spark\u306e\u30b0\u30e9\u30d5\u51e6\u7406\u30b7\u30b9\u30c6\u30e0\u3067\u3042\u308bGraphX\u3067\u4e0a\u56f3\u3092\u4f5c\u6210\u3059\u308b\u3068<\/span>\u671f\u5f85\u5165\u529b\u5f62\u5f0f\u306f\u4ee5\u4e0b\u306e\u3088\u3046\u306b\u306a\u308a\u307e\u3059\u3002<\/span><\/p>\n

0, 1, 1
\n0, 3, 3
\n1, 0, 1
\n1, 2, 2
\n1, 3, 1
\n2, 1, 2
\n2, 4, 4
\n3, 0, 3
\n3, 1, 1
\n3, 4, 4
\n4, 3, 4
\n4, 2, 4<\/code><\/p>\n

GraphX\u3067\u306fsource_id, destination_id, edge_value (\u63a5\u7d9a\u5143\u3001\u63a5\u7d9a\u5148\u3001\u8fba\u5024)\u3068\u3044\u3046\u5f62\u5f0f\u306b\u306a\u308a\u307e\u3059\u3002<\/span><\/p>\n

\u4e8c\u3064\u3092\u6bd4\u8f03\u3059\u308b\u3068Spark\u306e\u30b0\u30e9\u30d5\u30c7\u30fc\u30bf\u5f62\u5f0f\u306f\u8aad\u307f\u3084\u3059\u304f\u3001\u521d\u898b\u3067\u7406\u89e3\u3057\u3084\u3059\u3044\u3053\u3068\u304c\u308f\u304b\u308a\u307e\u3059\u3002\u3053\u306e\u5f62\u5f0f\u306b\u30d1\u30fc\u30b9\u3059\u308b\u306e\u3082\u6bd4\u8f03\u7684\u7c21\u5358\u3067\u3042\u308a\u307e\u3059\u3002<\/span>\u4e00\u65b9\u3067Giraph\u306f\u5404\u30ce\u30fc\u30c9\u304b\u3089\u306e\u8fba\u30921\u884c\u3067\u307e\u3068\u3081\u3066\u683c\u7d0d\u3057\u3066\u3044\u308b\u306e\u3067\u30c7\u30fc\u30bf\u884c\u304c\u5c11\u306a\u304f\u306a\u308a\u307e\u3059\u3002\u30b0\u30e9\u30d5\u30c7\u30fc\u30bf\u3092\u30a8\u30c3\u30b8\u30c7\u30fc\u30bf\u3068\u30ce\u30fc\u30c9\u30c7\u30fc\u30bf\u306b\u5206\u3051\u308b\u5fc5\u8981\u3082\u3042\u308a\u307e\u305b\u3093\u3002<\/span><\/p>\n

\u7d9a\u3044\u3066\u4f5c\u6210\u3057\u305f\u30b0\u30e9\u30d5\u30c7\u30fc\u30bf\u3092\u30d7\u30e9\u30a4\u30de\u30ea\u30b9\u30c8\u30ec\u30fc\u30b8\u306b\u683c\u7d0d\u3057\u307e\u3059\u3002\u30d7\u30e9\u30a4\u30de\u30ea\u30b9\u30c8\u30ec\u30fc\u30b8\u3068\u306fHDInsight\u306e\u30b9\u30c8\u30ec\u30fc\u30b8\u306b\u3042\u305f\u308bBLOB\u30b9\u30c8\u30ec\u30fc\u30b8\u4e0a\u3067\u52d5\u3044\u3066\u3044\u308bHDFS\u3067\u3059\u3002<\/span>\u4ee5\u4e0b\u306e\u30b3\u30de\u30f3\u30c9\u3067\u884c\u3044\u307e\u3059\u3002<\/p>\n

hadoop fs -copyFromLocal tiny_graph.txt \/example\/data\/tiny_graph.txt<\/code><\/p>\n

\"primary-storage\"<\/p>\n

\u30d7\u30e9\u30a4\u30de\u30ea\u30b9\u30c8\u30ec\u30fc\u30b8\u306f\u5404\u30af\u30e9\u30b9\u30bf\u30fc\u3054\u3068\u306b\u30b3\u30f3\u30c6\u30ca\u3067\u533a\u5225\u3055\u308c\u3066\u3044\u307e\u3059\u3002ryuo\u30af\u30e9\u30b9\u30bf\u30fc\u306e\u30b3\u30f3\u30c6\u30ca\u5185\u306e\/example\/data\/\u306b\u4f5c\u6210\u3057\u305f\u30b0\u30e9\u30d5\u304c\u683c\u7d0d\u3055\u308c\u305f\u3053\u3068\u304c\u5206\u304b\u308a\u307e\u3059\u3002<\/p>\n

\u6b21\u306bGiraph\u30d7\u30ed\u30b0\u30e9\u30e0\u306e\u30d1\u30e9\u30e1\u30fc\u30bf\u3068\u3057\u3066\u5fc5\u8981\u3067\u3042\u308b\u30b5\u30fc\u30d0\u30fc\u306eFQDN\u3092\u53d6\u5f97\u3057\u307e\u3059\u3002<\/span><\/p>\n

hostname -f<\/code><\/p>\n

watanabe@hn0-ryuou:~$ hostname -f
\nhn0-ryuou.bk4ryjujw10u3g3wen2ztys0zf.lx.internal.cloudapp.net<\/p>\n

FQDN\u3092\u53d6\u5f97\u3057\u305f\u3089\u3001\u4ee5\u4e0b\u306e\u30b3\u30de\u30f3\u30c9\u5f62\u5f0f\u3067Giraph\u30d7\u30ed\u30b0\u30e9\u30e0\u3092\u5b9f\u884c\u3057\u307e\u3059\u3002<\/p>\n

hadoop jar (jar\u30d5\u30a1\u30a4\u30eb) (\u30af\u30e9\u30b9\u540d) (\u95a2\u6570) -ca (\u30d8\u30c3\u30c9\u30ce\u30fc\u30c9) -vif (\u5165\u529b\u5f62\u5f0f) -vip (\u5165\u529b\u30c7\u30fc\u30bf) -vof (\u51fa\u529b\u5f62\u5f0f) -op (\u51fa\u529b\u5834\u6240) -w (\u30ef\u30fc\u30ab\u30fc\u6570) <\/code><\/p>\n

\u5b9f\u4f8b<\/p>\n

watanabe@hn0-ryuou:~$ hadoop jar \/usr\/hdp\/current\/giraph\/giraph-examples.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -ca mapred.job.tracker=HEADNODE:hn0-ryuou.bk4ryjujw10u3g3wen2ztys0zf.lx.internal.clou>dapp.net -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip \/example\/data\/tiny_graph.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op \/example\/output\/shortestpaths -w 2
\n<\/code><\/p>\n

\u51fa\u529b\u7d50\u679c
\n
\n16\/02\/17 02:21:44 INFO utils.ConfigurationUtils: No edge input format specified. Ensure your InputFormat does not require one.
\n16\/02\/17 02:21:44 INFO utils.ConfigurationUtils: No edge output format specified. Ensure your OutputFormat does not require one.
\n16\/02\/17 02:21:44 INFO utils.ConfigurationUtils: Setting custom argument [mapred.job.tracker] to [HEADNODE:hn0-ryuou.bk4ryjujw10u3g3wen2ztys0zf.lx.internal.cloudapp.net] in GiraphConfiguration
\n16\/02\/17 02:21:46 INFO job.GiraphJob: run: Since checkpointing is disabled (default), do not allow any task retries (setting mapred.map.max.attempts = 1, old value = 10)
\n16\/02\/17 02:21:47 INFO impl.TimelineClientImpl: Timeline service address: http:\/\/hn0-ryuou.bk4ryjujw10u3g3wen2ztys0zf.lx.internal.cloudapp.net:8188\/ws\/v1\/timeline\/
\n16\/02\/17 02:21:48 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm2
\n16\/02\/17 02:21:52 INFO mapreduce.JobSubmitter: number of splits:3
\n16\/02\/17 02:21:52 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1455616569359_0006
\n16\/02\/17 02:21:53 INFO impl.YarnClientImpl: Submitted application application_1455616569359_0006
\n16\/02\/17 02:21:54 INFO mapreduce.Job: The url to track the job: http:\/\/hn1-ryuou.bk4ryjujw10u3g3wen2ztys0zf.lx.internal.cloudapp.net:8088\/proxy\/application_1455616569359_0006\/
\n16\/02\/17 02:21:54 INFO job.GiraphJob: Tracking URL: http:\/\/hn1-ryuou.bk4ryjujw10u3g3wen2ztys0zf.lx.internal.cloudapp.net:8088\/proxy\/application_1455616569359_0006\/
\n16\/02\/17 02:21:54 INFO job.GiraphJob: Waiting for resources... Job will start only when it gets all 3 mappers
\n16\/02\/17 02:31:19 INFO job.HaltApplicationUtils$DefaultHaltInstructionsWriter: writeHaltInstructions: To halt after next superstep execute: 'bin\/halt-application --zkServer 10.0.0.5:22181 --zkNode \/_hadoopBsp\/job_1455616569359_0006\/_haltComputation'
\n16\/02\/17 02:31:19 INFO mapreduce.Job: Running job: job_1455616569359_0006
\n16\/02\/17 02:31:20 INFO mapreduce.Job: Job job_1455616569359_0006 running in uber mode : false
\n16\/02\/17 02:31:20 INFO mapreduce.Job: map 67% reduce 0%
\n16\/02\/17 02:51:22 INFO mapreduce.Job: map 100% reduce 0%
\n16\/02\/17 02:51:23 INFO mapreduce.Job: Job job_1455616569359_0006 failed with state KILLED due to: Kill job job_1455616569359_0006 received from watanabe (auth:SIMPLE) at 10.0.0.5
\nJob received Kill while in RUNNING state.
\n<\/code><\/p>\n

\u304a\u3063\u3068fail\u3057\u3066\u3057\u307e\u3044\u307e\u3057\u305f\u3002\u524d\u56de\u306eR\u691c\u8a3c\u306e\u969b\u306e\u30d0\u30b0\u3082\u8e0f\u307e\u3048\u3066\u63a8\u6e2c\u3059\u308b\u3068\u3069\u3046\u3084\u3089reduce\u306e\u51e6\u7406\u304c\u602a\u3057\u3044\u3067\u3059\u3002<\/p>\n

\"bug-screen\"<\/p>\n

\u30ad\u30e5\u30fc\u5f85\u3061\u3067\u53cd\u5fdc\u304c\u306a\u304f\u3001kill\u3055\u308c\u3066\u3044\u308b\u72b6\u614b\u3067\u3042\u308a\u307e\u3059\u3002<\/p>\n

tracking URL\u306b\u30a2\u30af\u30bb\u30b9\u3057\u3066\u3082\u3001\u63a5\u7d9a\u62d2\u5426\u3055\u308c\u3066\u3044\u307e\u3059\u3002<\/p>\n

\"tracking-unified-resource-locator\"<\/p>\n

reduce\u306f\u5404\u30ce\u30fc\u30c9\u306e\u7d50\u679c\u3092\u96c6\u8a08\u3059\u308b\u4f5c\u696d\u306a\u306e\u3067\u3001\u3082\u3057\u304b\u3059\u308b\u3068\u539f\u56e0\u306f\u30ef\u30fc\u30ab\u30fc\u30ce\u30fc\u30c9\u304c1\u500b\u306e\u5834\u5408\u3001\u52d5\u304b\u306a\u3044\u306e\u3067\u306f\u306a\u3044\u304b\u3068\u63a8\u6e2c\u3057\u307e\u3059\u3002(*\u9023\u8f09\u7b2c\u4e00\u56de<\/a>\u30af\u30e9\u30b9\u30bf\u4f5c\u6210\u306e\u30ce\u30fc\u30c9\u6599\u91d1\u30ec\u30d9\u30eb\u3092\u53c2\u7167)<\/p>\n

\u305d\u3053\u3067\u30c0\u30c3\u30b7\u30e5\u30dc\u30fc\u30c9\u306e\u300c\u3059\u3079\u3066\u306e\u30ea\u30bd\u30fc\u30b9\u300d\u2192\u300c\u30af\u30e9\u30b9\u30bf\u540d(ryuou)\u300d\u2192\u300c\u30af\u30e9\u30b9\u30bf\u306e\u30b9\u30b1\u30fc\u30eb\u8a2d\u5b9a\u300d\u2192\u300c\u30ef\u30fc\u30ab\u30fc\u30ce\u30fc\u30c9\u306e\u6570\u300d\u3092\u8a2d\u5b9a\u3057\u76f4\u3057\u307e\u3059\u3002<\/p>\n

\"cluster-setting\"<\/p>\n

\u4ee5\u524d\u306f1\u30ce\u30fc\u30c9\u3067\u691c\u8a3c\u3092\u884c\u3063\u3066\u3044\u305f\u304c\u3001\u30ce\u30fc\u30c9\u6570\u30922\u500b\u306b\u66f4\u65b0\u3057\u307e\u3057\u305f<\/span>\u3002<\/span><\/p>\n

\"new-setting\"<\/p>\n

\u518d\u6311\u6226\u3057\u3066\u307f\u307e\u3059<\/p>\n

watanabe@hn0-ryuou:~$ hadoop jar \/usr\/hdp\/current\/giraph\/giraph-examples.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -ca mapred.job.tracker=HEADNODE:hn0-ryuou.bk4ryjujw10u3g3wen2ztys0zf.lx.internal.cloudapp.net -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip \/example\/data\/tiny_graph.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op \/example\/output\/shortestpaths -w 2
\n<\/code><\/p>\n

\u51fa\u529b<\/p>\n

16\/02\/17 05:37:32 INFO utils.ConfigurationUtils: No edge input format specified. Ensure your InputFormat does not require one.
\n16\/02\/17 05:37:32 INFO utils.ConfigurationUtils: No edge output format specified. Ensure your OutputFormat does not require one.
\n16\/02\/17 05:37:32 INFO utils.ConfigurationUtils: Setting custom argument [mapred.job.tracker] to [HEADNODE:hn0-ryuou.bk4ryjujw10u3g3wen2ztys0zf.lx.internal.cloudapp.net] in GiraphConfiguration
\n16\/02\/17 05:37:34 INFO job.GiraphJob: run: Since checkpointing is disabled (default), do not allow any task retries (setting mapred.map.max.attempts = 1, old value = 10)<\/code><\/p>\n

Exception in thread \"main\" org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory wasb:\/\/ryuou@akibaline.blob.core.windows.net\/example\/output\/shortestpaths already exists<\/p>\n

\u518d\u5ea6fail\u3044\u305f\u3057\u307e\u3057\u305f\u304c\u3001\u300c\u3059\u3067\u306b\u30a2\u30a6\u30c8\u30d7\u30c3\u30c8\u30d5\u30a1\u30a4\u30eb\u304c\u5b58\u5728\u3059\u308b\u300d\u3068\u3044\u3046\u30a8\u30e9\u30fc\u3067\u3059\u306e\u3067\u3001\u518d\u5b9f\u884c\u306e\u969b\u306f\u751f\u6210\u6e08\u307f\u306e\u51fa\u529b\u30c7\u30a3\u30ec\u30af\u30c8\u30ea\u3092\u6d88\u3059\u304b\u3001\u51fa\u529b\u30c7\u30a3\u30ec\u30af\u30c8\u30ea\u5148\u3092\u5909\u66f4\u3059\u308b\u3053\u3068\u304c\u5fc5\u8981\u3067\u3059\u3002<\/p>\n

\u518d\u3005\u6311\u6226<\/p>\n

watanabe@hn0-ryuou:~$ hadoop jar \/usr\/hdp\/current\/giraph\/giraph-examples.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -ca mapred.job.tracker=HEADNODE:hn0-ryuou.bk4ryjujw10u3g3wen2ztys0zf.lx.internal.cloudapp.net -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip \/example\/data\/tiny_graph.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op \/example\/output\/shortestpaths -w 2
\n<\/code><\/p>\n

\u51fa\u529b<\/p>\n

16\/02\/17 03:58:09 INFO utils.ConfigurationUtils: No edge input format specified. Ensure your InputFormat does not require one.
\n16\/02\/17 03:58:09 INFO utils.ConfigurationUtils: No edge output format specified. Ensure your OutputFormat does not require one.
\n16\/02\/17 03:58:09 INFO utils.ConfigurationUtils: Setting custom argument [mapred.job.tracker] to [HEADNODE:hn0-ryuou.bk4ryjujw10u3g3wen2ztys0zf.lx.internal.cloudapp.net] in GiraphConfiguration
\n16\/02\/17 03:58:10 INFO job.GiraphJob: run: Since checkpointing is disabled (default), do not allow any task retries (setting mapred.map.max.attempts = 1, old value = 10)
\n16\/02\/17 03:58:12 INFO impl.TimelineClientImpl: Timeline service address: http:\/\/hn0-ryuou.bk4ryjujw10u3g3wen2ztys0zf.lx.internal.cloudapp.net:8188\/ws\/v1\/timeline\/
\n16\/02\/17 03:58:13 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm2
\n16\/02\/17 03:58:16 INFO mapreduce.JobSubmitter: number of splits:3
\n16\/02\/17 03:58:16 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1455616569359_0007
\n16\/02\/17 03:58:17 INFO impl.YarnClientImpl: Submitted application application_1455616569359_0007
\n16\/02\/17 03:58:17 INFO mapreduce.Job: The url to track the job: http:\/\/hn1-ryuou.bk4ryjujw10u3g3wen2ztys0zf.lx.internal.cloudapp.net:8088\/proxy\/application_1455616569359_0007\/
\n16\/02\/17 03:58:17 INFO job.GiraphJob: Tracking URL: http:\/\/hn1-ryuou.bk4ryjujw10u3g3wen2ztys0zf.lx.internal.cloudapp.net:8088\/proxy\/application_1455616569359_0007\/
\n16\/02\/17 03:58:17 INFO job.GiraphJob: Waiting for resources... Job will start only when it gets all 3 mappers
\n16\/02\/17 03:58:39 INFO job.HaltApplicationUtils$DefaultHaltInstructionsWriter: writeHaltInstructions: To halt after next superstep execute: 'bin\/halt-application --zkServer 10.0.0.5:22181 --zkNode \/_hadoopBsp\/job_1455616569359_0007\/_haltComputation'
\n16\/02\/17 03:58:39 INFO mapreduce.Job: Running job: job_1455616569359_0007
\n16\/02\/17 03:58:40 INFO mapreduce.Job: Job job_1455616569359_0007 running in uber mode : false
\n16\/02\/17 03:58:40 INFO mapreduce.Job: map 33% reduce 0%
\n16\/02\/17 03:58:47 INFO mapreduce.Job: map 67% reduce 0%
\n16\/02\/17 03:58:57 INFO mapreduce.Job: map 100% reduce 0%
\n16\/02\/17 03:59:06 INFO mapreduce.Job: Job job_1455616569359_0007 completed successfully
\n16\/02\/17 03:59:06 INFO mapreduce.Job: Counters: 53
\n<\/code><\/p>\n

\u51e6\u7406\u304c\u8a70\u307e\u308a\u307e\u305b\u3093\uff01\u3084\u3063\u305f\uff01\u3053\u308c\u3067\u6b63\u5e38\u5b8c\u4e86\u3067\u3059\u3002<\/p>\n

\u6b63\u5e38\u30b5\u30a4\u30f3<\/p>\n

Giraph Stats
\nAggregate edges=12
\nAggregate finished vertices=5
\nAggregate sent message message bytes=410
\nAggregate sent messages=12
\nAggregate vertices=5
\nCurrent master task partition=0
\nCurrent workers=2
\nLast checkpointed superstep=0
\nSent message bytes=0
\nSent messages=0
\nSuperstep=4
\nGiraph Timers
\nInitialize (ms)=13670
\nInput superstep (ms)=307
\nSetup (ms)=62
\nShutdown (ms)=9116
\nSuperstep 0 SimpleShortestPathsComputation (ms)=109
\nSuperstep 1 SimpleShortestPathsComputation (ms)=125
\nSuperstep 2 SimpleShortestPathsComputation (ms)=88
\nSuperstep 3 SimpleShortestPathsComputation (ms)=72
\nTotal (ms)=9881
\n<\/code><\/p>\n

\u51fa\u529b\u3055\u308c\u305f\u7d50\u679c\u3092\u898b\u3066\u307f\u307e\u3059\u3002Hadoop\u3084Spark\u306a\u3069\u306e\u4e26\u5217\u51e6\u7406\u30b7\u30b9\u30c6\u30e0\u306e\u8a2d\u5b9a\u3067\u306f\u51fa\u529b\u30d5\u30a1\u30a4\u30eb\u306f\u5404\u30ce\u30fc\u30c9\u3054\u3068\u306b\u5410\u304d\u51fa\u3055\u308c\u308b\u305f\u3081\u3001text\u30b3\u30de\u30f3\u30c9\u3092\u4f7f\u3044HDFS\u4e0a\u3067\u307e\u3068\u3081\u3066\u304b\u3089\u898b\u308b\u304b\u3001get\u30b3\u30de\u30f3\u30c9\u5f8c\u306b\u30ed\u30fc\u30ab\u30eb\u3067cat\u3057\u307e\u3059\u3002<\/p>\n

watanabe@hn0-ryuou:~$ hadoop fs -text \/example\/output\/shortestpaths\/*
\n0 1.0
\n4 5.0
\n2 2.0
\n1 0.0
\n3 1.0<\/code><\/p>\n

ID1\u30ce\u30fc\u30c9\u304b\u3089\u306e\u6700\u77ed\u7d4c\u8def\u30b0\u30e9\u30d5\u306e\u56f3<\/p>\n

\"output-graph\"<\/p>\n

\u4ee5\u4e0a\u3067\u3059\u3002Giraph\u306e\u516c\u5f0f\u30c1\u30e5\u30fc\u30c8\u30ea\u30a2\u30eb\u306f\u3053\u3061\u3089<\/a>\u3067\u3059\u3002<\/p>\n

\u3053\u3053\u307e\u3067\u306e\u611f\u60f3<\/h2>\n

\u4eca\u56de\u306e\u30a8\u30e9\u30fc\u306f\u4ed6\u6240\u3067\u30b3\u30f3\u30d1\u30a4\u30eb\u6e08\u307f\u306e\u30b5\u30f3\u30d7\u30ebjar\u30d5\u30a1\u30a4\u30eb\u3092\u4f7f\u7528\u3057\u305f\u3053\u3068\u3067\u767a\u751f\u3057\u307e\u3057\u305f\u3002\u65e2\u5b58\u306ejar\u30d7\u30ed\u30b0\u30e9\u30e0\u306e\u4f5c\u52d5\u7bc4\u56f2\u306e\u74b0\u5883\u3092\u6574\u5099\u3059\u308b\u3053\u3068\u304c\u91cd\u8981\u3067\u3057\u305f\u3002\u4eee\u306b\u30bd\u30fc\u30b9\u30b3\u30fc\u30c9\u304c\u63d0\u4f9b\u3055\u308c\u3066\u3044\u305f\u306e\u3067\u3042\u308c\u3070\u3001\u81ea\u5206\u306e\u74b0\u5883\u306b\u3082\u5bfe\u5fdc\u3059\u308b\u3088\u3046\u30d7\u30ed\u30b0\u30e9\u30e0\u3092\u4fee\u6b63\u3057\u3001jar\u30d5\u30a1\u30a4\u30eb\u3092\u30b3\u30f3\u30d1\u30a4\u30eb\u3057\u76f4\u3059\u3053\u3068\u3082\u3067\u304d\u305f\u306e\u304b\u306a\u3068\u601d\u3044\u307e\u3059\u3002\u3044\u305a\u308c\u306b\u3057\u3066\u3082\u30a8\u30e9\u30fc\u306e\u5185\u5bb9\u3092\u3088\u304f\u898b\u3066\u3001\u9069\u5207\u306b\u5bfe\u5fdc\u3059\u308b\u3053\u3068\u304c\u5927\u5207\u3067\u3059\u3002<\/p>\n

\u6b21\u56de\u306fazure\u306e\u30af\u30e9\u30b9\u30bf\u30fc\u3092\u30b3\u30de\u30f3\u30c9\u30e9\u30a4\u30f3\u3067\u64cd\u4f5c\u3067\u304d\u308bCLI(\u30b3\u30de\u30f3\u30c9\u30e9\u30a4\u30f3\u30a4\u30f3\u30bf\u30fc\u30d5\u30a7\u30fc\u30b9)\u3092\u4f7f\u3063\u3066\u307f\u307e\u3059\u3002<\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3\u6e21\u8fba\u3067\u3059\u3002\u4eca\u56de\u306f\u9023\u8f09\u306e\u7b2c\u4e09\u56de\u3067\u3001Hadoop\u30a8\u30b3\u30b7\u30b9\u30c6\u30e0\u306e\u4e00\u3064\u3067\u3042\u308b\u4e26\u5217\u30b0\u30e9\u30d5\u51e6\u7406\u57fa\u76e4\u300cGiraph\u300d\u3092Azure\u4e0a\u3067\u52d5\u304b\u3057\u3066\u307f\u307e\u3059\u3002\u30c7\u30fc\u30bf\u304c\u30cf\u30c1\u306e\u5de3\u306e\u3088\u3046\u306b\u96c6\u307e\u308bApache Hive\u306e\u6b21\u306f\u30ad\u30ea\u30f3\u306e […]<\/p>\n","protected":false},"author":1,"featured_media":13095,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"content-type":"","footnotes":""},"categories":[58,16],"tags":[66,65],"class_list":["post-13093","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-azure","category-author","tag-hdinsight","tag-microsoft-azure"],"acf":[],"yoast_head":"\nHadoop\u3092Microsoft Azure\u4e0a\u3067\u52d5\u304b\u3057\u3066\u307f\u308b\uff01\u3000(3\/4) - Tech Blog\uff5c\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3<\/title>\n<meta name=\"description\" content=\"Azure, \u8457\u8005\uff08Author\uff09 |\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3\u6e21\u8fba\u3067\u3059\u3002\u4eca\u56de\u306f\u9023\u8f09\u306e\u7b2c\u4e09\u56de\u3067\u3001Hadoop\u30a8\u30b3\u30b7\u30b9\u30c6\u30e0\u306e\u4e00\u3064\u3067\u3042\u308b\u4e26\u5217\u30b0\u30e9\u30d5\u51e6\u7406\u57fa\u76e4\u300cGiraph\u300d\u3092Azure\u4e0a\u3067\u52d5\u304b\u3057\u3066\u307f\u307e\u3059\u3002\u30c7\u30fc\u30bf\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/13093\" \/>\n<meta property=\"og:locale\" content=\"ja_JP\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Hadoop\u3092Microsoft Azure\u4e0a\u3067\u52d5\u304b\u3057\u3066\u307f\u308b\uff01\u3000(3\/4) - Tech Blog\uff5c\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3\" \/>\n<meta property=\"og:description\" content=\"Azure, \u8457\u8005\uff08Author\uff09 |\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3\u6e21\u8fba\u3067\u3059\u3002\u4eca\u56de\u306f\u9023\u8f09\u306e\u7b2c\u4e09\u56de\u3067\u3001Hadoop\u30a8\u30b3\u30b7\u30b9\u30c6\u30e0\u306e\u4e00\u3064\u3067\u3042\u308b\u4e26\u5217\u30b0\u30e9\u30d5\u51e6\u7406\u57fa\u76e4\u300cGiraph\u300d\u3092Azure\u4e0a\u3067\u52d5\u304b\u3057\u3066\u307f\u307e\u3059\u3002\u30c7\u30fc\u30bf\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/13093\" \/>\n<meta property=\"og:site_name\" content=\"Tech Blog\uff5c\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/creationline\" \/>\n<meta property=\"article:published_time\" content=\"2016-03-16T04:53:46+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2016-11-05T05:18:22+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/giraph-azur.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"373\" \/>\n\t<meta property=\"og:image:height\" content=\"199\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"admin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@creationline\" \/>\n<meta name=\"twitter:site\" content=\"@creationline\" \/>\n<meta name=\"twitter:label1\" content=\"\u57f7\u7b46\u8005\" \/>\n\t<meta name=\"twitter:data1\" content=\"admin\" \/>\n\t<meta name=\"twitter:label2\" content=\"\u63a8\u5b9a\u8aad\u307f\u53d6\u308a\u6642\u9593\" \/>\n\t<meta name=\"twitter:data2\" content=\"5\u5206\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cloudnative\\\/azure\\\/13093#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cloudnative\\\/azure\\\/13093\"},\"author\":{\"name\":\"admin\",\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/#\\\/schema\\\/person\\\/7d923d1c017568a1a5e66d7bb1c8764a\"},\"headline\":\"Hadoop\u3092Microsoft Azure\u4e0a\u3067\u52d5\u304b\u3057\u3066\u307f\u308b\uff01\u3000(3\\\/4)\",\"datePublished\":\"2016-03-16T04:53:46+00:00\",\"dateModified\":\"2016-11-05T05:18:22+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cloudnative\\\/azure\\\/13093\"},\"wordCount\":125,\"image\":{\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cloudnative\\\/azure\\\/13093#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cms_x3GWkuX\\\/wp-content\\\/uploads\\\/2016\\\/03\\\/giraph-azur.jpg\",\"keywords\":[\"HDInsight\",\"Microsoft Azure\"],\"articleSection\":[\"Azure\",\"\u8457\u8005\uff08Author\uff09\"],\"inLanguage\":\"ja\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cloudnative\\\/azure\\\/13093\",\"url\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cloudnative\\\/azure\\\/13093\",\"name\":\"Hadoop\u3092Microsoft Azure\u4e0a\u3067\u52d5\u304b\u3057\u3066\u307f\u308b\uff01\u3000(3\\\/4) - Tech Blog\uff5c\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cloudnative\\\/azure\\\/13093#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cloudnative\\\/azure\\\/13093#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cms_x3GWkuX\\\/wp-content\\\/uploads\\\/2016\\\/03\\\/giraph-azur.jpg\",\"datePublished\":\"2016-03-16T04:53:46+00:00\",\"dateModified\":\"2016-11-05T05:18:22+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/#\\\/schema\\\/person\\\/7d923d1c017568a1a5e66d7bb1c8764a\"},\"description\":\"Azure, \u8457\u8005\uff08Author\uff09 |\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3\u6e21\u8fba\u3067\u3059\u3002\u4eca\u56de\u306f\u9023\u8f09\u306e\u7b2c\u4e09\u56de\u3067\u3001Hadoop\u30a8\u30b3\u30b7\u30b9\u30c6\u30e0\u306e\u4e00\u3064\u3067\u3042\u308b\u4e26\u5217\u30b0\u30e9\u30d5\u51e6\u7406\u57fa\u76e4\u300cGiraph\u300d\u3092Azure\u4e0a\u3067\u52d5\u304b\u3057\u3066\u307f\u307e\u3059\u3002\u30c7\u30fc\u30bf\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cloudnative\\\/azure\\\/13093#breadcrumb\"},\"inLanguage\":\"ja\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cloudnative\\\/azure\\\/13093\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"ja\",\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cloudnative\\\/azure\\\/13093#primaryimage\",\"url\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cms_x3GWkuX\\\/wp-content\\\/uploads\\\/2016\\\/03\\\/giraph-azur.jpg\",\"contentUrl\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cms_x3GWkuX\\\/wp-content\\\/uploads\\\/2016\\\/03\\\/giraph-azur.jpg\",\"width\":373,\"height\":199},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cloudnative\\\/azure\\\/13093#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"HOME\",\"item\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"\u30af\u30e9\u30a6\u30c9\u30cd\u30a4\u30c6\u30a3\u30d6\",\"item\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cloudnative\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Azure\",\"item\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cloudnative\\\/azure\"},{\"@type\":\"ListItem\",\"position\":4,\"name\":\"Hadoop\u3092Microsoft Azure\u4e0a\u3067\u52d5\u304b\u3057\u3066\u307f\u308b\uff01\u3000(3\\\/4)\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/#website\",\"url\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/\",\"name\":\"Tech Blog\uff5c\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3\",\"description\":\"\u30a2\u30b8\u30e3\u30a4\u30eb\uff06DevOps\u3001\u30af\u30e9\u30a6\u30c9\u30cd\u30a4\u30c6\u30a3\u30d6\u3001AI\uff06LLM\u306e\u5148\u7aef\u6280\u8853\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"ja\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/#\\\/schema\\\/person\\\/7d923d1c017568a1a5e66d7bb1c8764a\",\"name\":\"admin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"ja\",\"@id\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cms_x3GWkuX\\\/wp-content\\\/uploads\\\/2021\\\/12\\\/avatar.png\",\"url\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cms_x3GWkuX\\\/wp-content\\\/uploads\\\/2021\\\/12\\\/avatar.png\",\"contentUrl\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/cms_x3GWkuX\\\/wp-content\\\/uploads\\\/2021\\\/12\\\/avatar.png\",\"caption\":\"admin\"},\"url\":\"https:\\\/\\\/www.creationline.com\\\/tech-blog\\\/author\\\/admin\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Hadoop\u3092Microsoft Azure\u4e0a\u3067\u52d5\u304b\u3057\u3066\u307f\u308b\uff01\u3000(3\/4) - Tech Blog\uff5c\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3","description":"Azure, \u8457\u8005\uff08Author\uff09 |\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3\u6e21\u8fba\u3067\u3059\u3002\u4eca\u56de\u306f\u9023\u8f09\u306e\u7b2c\u4e09\u56de\u3067\u3001Hadoop\u30a8\u30b3\u30b7\u30b9\u30c6\u30e0\u306e\u4e00\u3064\u3067\u3042\u308b\u4e26\u5217\u30b0\u30e9\u30d5\u51e6\u7406\u57fa\u76e4\u300cGiraph\u300d\u3092Azure\u4e0a\u3067\u52d5\u304b\u3057\u3066\u307f\u307e\u3059\u3002\u30c7\u30fc\u30bf","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/13093","og_locale":"ja_JP","og_type":"article","og_title":"Hadoop\u3092Microsoft Azure\u4e0a\u3067\u52d5\u304b\u3057\u3066\u307f\u308b\uff01\u3000(3\/4) - Tech Blog\uff5c\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3","og_description":"Azure, \u8457\u8005\uff08Author\uff09 |\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3\u6e21\u8fba\u3067\u3059\u3002\u4eca\u56de\u306f\u9023\u8f09\u306e\u7b2c\u4e09\u56de\u3067\u3001Hadoop\u30a8\u30b3\u30b7\u30b9\u30c6\u30e0\u306e\u4e00\u3064\u3067\u3042\u308b\u4e26\u5217\u30b0\u30e9\u30d5\u51e6\u7406\u57fa\u76e4\u300cGiraph\u300d\u3092Azure\u4e0a\u3067\u52d5\u304b\u3057\u3066\u307f\u307e\u3059\u3002\u30c7\u30fc\u30bf","og_url":"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/13093","og_site_name":"Tech Blog\uff5c\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3","article_publisher":"https:\/\/www.facebook.com\/creationline","article_published_time":"2016-03-16T04:53:46+00:00","article_modified_time":"2016-11-05T05:18:22+00:00","og_image":[{"width":373,"height":199,"url":"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/giraph-azur.jpg","type":"image\/jpeg"}],"author":"admin","twitter_card":"summary_large_image","twitter_creator":"@creationline","twitter_site":"@creationline","twitter_misc":{"\u57f7\u7b46\u8005":"admin","\u63a8\u5b9a\u8aad\u307f\u53d6\u308a\u6642\u9593":"5\u5206"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/13093#article","isPartOf":{"@id":"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/13093"},"author":{"name":"admin","@id":"https:\/\/www.creationline.com\/tech-blog\/#\/schema\/person\/7d923d1c017568a1a5e66d7bb1c8764a"},"headline":"Hadoop\u3092Microsoft Azure\u4e0a\u3067\u52d5\u304b\u3057\u3066\u307f\u308b\uff01\u3000(3\/4)","datePublished":"2016-03-16T04:53:46+00:00","dateModified":"2016-11-05T05:18:22+00:00","mainEntityOfPage":{"@id":"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/13093"},"wordCount":125,"image":{"@id":"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/13093#primaryimage"},"thumbnailUrl":"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/giraph-azur.jpg","keywords":["HDInsight","Microsoft Azure"],"articleSection":["Azure","\u8457\u8005\uff08Author\uff09"],"inLanguage":"ja"},{"@type":"WebPage","@id":"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/13093","url":"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/13093","name":"Hadoop\u3092Microsoft Azure\u4e0a\u3067\u52d5\u304b\u3057\u3066\u307f\u308b\uff01\u3000(3\/4) - Tech Blog\uff5c\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3","isPartOf":{"@id":"https:\/\/www.creationline.com\/tech-blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/13093#primaryimage"},"image":{"@id":"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/13093#primaryimage"},"thumbnailUrl":"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/giraph-azur.jpg","datePublished":"2016-03-16T04:53:46+00:00","dateModified":"2016-11-05T05:18:22+00:00","author":{"@id":"https:\/\/www.creationline.com\/tech-blog\/#\/schema\/person\/7d923d1c017568a1a5e66d7bb1c8764a"},"description":"Azure, \u8457\u8005\uff08Author\uff09 |\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3\u6e21\u8fba\u3067\u3059\u3002\u4eca\u56de\u306f\u9023\u8f09\u306e\u7b2c\u4e09\u56de\u3067\u3001Hadoop\u30a8\u30b3\u30b7\u30b9\u30c6\u30e0\u306e\u4e00\u3064\u3067\u3042\u308b\u4e26\u5217\u30b0\u30e9\u30d5\u51e6\u7406\u57fa\u76e4\u300cGiraph\u300d\u3092Azure\u4e0a\u3067\u52d5\u304b\u3057\u3066\u307f\u307e\u3059\u3002\u30c7\u30fc\u30bf","breadcrumb":{"@id":"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/13093#breadcrumb"},"inLanguage":"ja","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/13093"]}]},{"@type":"ImageObject","inLanguage":"ja","@id":"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/13093#primaryimage","url":"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/giraph-azur.jpg","contentUrl":"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2016\/03\/giraph-azur.jpg","width":373,"height":199},{"@type":"BreadcrumbList","@id":"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure\/13093#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"HOME","item":"https:\/\/www.creationline.com\/tech-blog"},{"@type":"ListItem","position":2,"name":"\u30af\u30e9\u30a6\u30c9\u30cd\u30a4\u30c6\u30a3\u30d6","item":"https:\/\/www.creationline.com\/tech-blog\/cloudnative"},{"@type":"ListItem","position":3,"name":"Azure","item":"https:\/\/www.creationline.com\/tech-blog\/cloudnative\/azure"},{"@type":"ListItem","position":4,"name":"Hadoop\u3092Microsoft Azure\u4e0a\u3067\u52d5\u304b\u3057\u3066\u307f\u308b\uff01\u3000(3\/4)"}]},{"@type":"WebSite","@id":"https:\/\/www.creationline.com\/tech-blog\/#website","url":"https:\/\/www.creationline.com\/tech-blog\/","name":"Tech Blog\uff5c\u30af\u30ea\u30a8\u30fc\u30b7\u30e7\u30f3\u30e9\u30a4\u30f3","description":"\u30a2\u30b8\u30e3\u30a4\u30eb\uff06DevOps\u3001\u30af\u30e9\u30a6\u30c9\u30cd\u30a4\u30c6\u30a3\u30d6\u3001AI\uff06LLM\u306e\u5148\u7aef\u6280\u8853","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.creationline.com\/tech-blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"ja"},{"@type":"Person","@id":"https:\/\/www.creationline.com\/tech-blog\/#\/schema\/person\/7d923d1c017568a1a5e66d7bb1c8764a","name":"admin","image":{"@type":"ImageObject","inLanguage":"ja","@id":"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2021\/12\/avatar.png","url":"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2021\/12\/avatar.png","contentUrl":"https:\/\/www.creationline.com\/tech-blog\/cms_x3GWkuX\/wp-content\/uploads\/2021\/12\/avatar.png","caption":"admin"},"url":"https:\/\/www.creationline.com\/tech-blog\/author\/admin"}]}},"_links":{"self":[{"href":"https:\/\/www.creationline.com\/tech-blog\/wp-json\/wp\/v2\/posts\/13093","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.creationline.com\/tech-blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.creationline.com\/tech-blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.creationline.com\/tech-blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.creationline.com\/tech-blog\/wp-json\/wp\/v2\/comments?post=13093"}],"version-history":[{"count":15,"href":"https:\/\/www.creationline.com\/tech-blog\/wp-json\/wp\/v2\/posts\/13093\/revisions"}],"predecessor-version":[{"id":13241,"href":"https:\/\/www.creationline.com\/tech-blog\/wp-json\/wp\/v2\/posts\/13093\/revisions\/13241"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.creationline.com\/tech-blog\/wp-json\/wp\/v2\/media\/13095"}],"wp:attachment":[{"href":"https:\/\/www.creationline.com\/tech-blog\/wp-json\/wp\/v2\/media?parent=13093"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.creationline.com\/tech-blog\/wp-json\/wp\/v2\/categories?post=13093"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.creationline.com\/tech-blog\/wp-json\/wp\/v2\/tags?post=13093"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}