μΌ | μ | ν | μ | λͺ© | κΈ | ν |
---|---|---|---|---|---|---|
1 | 2 | 3 | 4 | |||
5 | 6 | 7 | 8 | 9 | 10 | 11 |
12 | 13 | 14 | 15 | 16 | 17 | 18 |
19 | 20 | 21 | 22 | 23 | 24 | 25 |
26 | 27 | 28 | 29 | 30 | 31 |
- 리λ μ€ RPM
- μ€μλΈ μ€μΉ
- kudu
- μνμΉ μ€νν¬
- BIGDATA
- rpmbuild
- νλ‘
- μ€μΉΌλΌ λΆ
- μ€μλΈ
- λ¦Ώμ½λ
- Apache spark
- HBase
- LeetCode
- elastic stack
- νλ‘μμ½μμ€ν
- μλΌμ€ν±μμΉ
- ansible
- λ°μ΄ν°λΆμμ€μ λ¬Έκ°
- hadoop
- νμ΄λΈ
- standalone
- airflow
- νλ‘μ€μΉ
- aws ccp
- CLF-C02
- ν΄ μ€μΉ
- μ€νν¬
- 리λ μ€
- λΉ λ°μ΄ν°
- μ±κΈμλ²
- Today
- Total
data_lab
Hadoop HA μ€ν λ³Έλ¬Έ
μλ² κ΅¬μ±λ μμ
Zookeeper | HDFS | YARN | MapReduce | |
Master | quorumpeermain | journal node /Name Node (active) |
ResourceManager | Job Histroyserver |
Standby Master | quorumpeer | journal node/ Name Node (stand by) |
ResourceManager | |
Worker 1 | quorumpeer | journal node/ Data Node 1 |
NodeManager | |
Worker 2 | Data Node 2 | NodeManager | ||
Worker 3 | Data Node 3 | NodeManager |
1. zookeeper μ€ν
- κ° μλ² zkServer.sh start
- quorumpeer, quorumpeermain μ€ν λ¨
- leader, follower μ μ μ λλ€μ
2. journal node μ€ν - zkμλ² μ€νν μλ²μμ μ€νν¨ (Namenode HAλ₯Ό μν¨)
- μ€νν΄μΌ 8485 ν¬νΈ μ΄λ¦Ό. μ€νμνλ©΄ λ€μλ Έλ ν¬λ§·μ μλ¬λΈ
3. active Namenode λ₯Ό μ€νν μλ²μμ μμ
- Namenode resset
- Namenode start
4. standby Namenode μ€νν μλ²μμ μμ
- active Namenodeμ metadata 볡μ¬
- hdfs namenode -bootstrapStandby
- Namenode start
5. namenode μλ²μ zookeeper failcontroller μ€ν
6. Worker 1-3 μλ²μμ Datanode μ€ν
7. RM, NM κ° μλ²μμ μ€ν
- Resource Manager λ HA κ΅¬μ± κ°λ₯ν¨
8. Master μλ²μμ MR job historyserver λ°λͺ¬ μ€ν
'BIGDATA > νλ‘μμ½μμ€ν ' μΉ΄ν κ³ λ¦¬μ λ€λ₯Έ κΈ
hdfs ꡬμ±μμ secondary namenode μ HAλ‘ λ stand by namenodeμ μν μ μ°¨μ΄ (0) | 2023.04.06 |
---|---|
Grafana + Prometheus λͺ¨λν°λ§ κ΅¬μΆ (0) | 2022.05.25 |
[Hue] Hue 4.10.0 μ€μΉ κ°μ΄λ (1) | 2022.05.13 |
Imapa VS Hive (0) | 2022.03.30 |
[kudu] kudu clusterλͺ¨λλ‘ μ€ν (0) | 2022.03.29 |