[하둡교육 3일차] Pig
Pig set up
1) [root@localhost root]# wget http://www.dazum.net/hadoop/pig-0.10.0.tar.gz : 소스를 가져온다
2) [root@localhost root]# tar xvf pig-0.10.0.tar.gz : 압축을 푼다
3) [root@localhost root]# ln -s pig-0.10.0 pig : 심볼릭 링크 걸어준다 버전 관리를 위해)
4) [root@localhost root]# vi /etc/profile : 환경 셋팅을 해준다. root 기준임.
export PIG_HOME=/home/root/pig
export PIG_CLASSPATH=$HADOOP_HOME/conf
pathmunge /home/root/pig/bin
[root@localhost root]# source /etc/profile : 프로파일을 적용 해준다.
5) pig
[root@localhost root]# pig : pig 실행
Warning: $HADOOP_HOME is deprecated.
2013-06-05 10:29:49,282 [main] INFO org.apache.pig.Main - Apache Pig version 0.10.0 (r1328203) compiled Apr 19 2012, 22:54:12
2013-06-05 10:29:49,282 [main] INFO org.apache.pig.Main - Logging error messages to: /home/root/pig_1370395789279.log
2013-06-05 10:29:49,479 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://node01:9000/
2013-06-05 10:29:49,647 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: node01:9001
grunt>
some node
[root@localhost ~]# hadoop dfsadmin -safemode leave
prepare example
1) [root@localhost ~]# vi pig.txt
aaa,100
bbb.200
ccc,300
bbb,400
2) [root@localhost ~]# hadoop fs -put pig.txt .
3) [root@localhost ~]# hadoop fs -ls
Warning: $HADOOP_HOME is deprecated.
Found 1 items
-rw-r--r-- 2 root supergroup 32 2013-06-05 11:07 /user/root/pig.txt
pig
grunt> ls
hdfs://node01:9000/user/root/pig.txt<r 2> 32
grunt> cat pig.txt
aaa,100
bbb.200
ccc,300
bbb,400
grunt>
grunt> cp pig.txt pig2.txt
grunt> ls
hdfs://node01:9000/user/root/pig.txt<r 2> 32
hdfs://node01:9000/user/root/pig2.txt<r 2> 32
grunt> illustrate c;
여러가지 커맨드들이 있으며, pig는 map Reduce를 간단히 처리 할 수 있도록 해준다.
교육 자체가..실습보단 하는 걸 보여주었다..아쉽게 직접해봐야 포스팅도 알찰텐데..ㅠ_ㅠ
직접 해보면서 정리를 해야 할 것 같다..ㅎㅎ
-끝-