报错：
XXX is running 64299008B beyond the ‘PHYSICAL’ memory limit. Current usage: 2.1 GB of 2 GB physical memory used; 4.3 GB of 4.2 GB virtual memory used. Killing container.
set yarn.scheduler.minimum-allocation-mb;
±-------------------------------------------+
| set |
±-------------------------------------------+
| yarn.scheduler.minimum-allocation-mb=1024 |
±-------------------------------------------+
1 row selected (0.088 seconds)
set yarn.scheduler.maximum-allocation-mb;
±--------------------------------------------+
| set |
±--------------------------------------------+
| yarn.scheduler.maximum-allocation-mb=65536 |
±--------------------------------------------+
1 row selected (0.007 seconds)
set mapreduce.map.memory.mb;
±------------------------------+
| set |
±------------------------------+
| mapreduce.map.memory.mb=2048 |
±------------------------------+
1 row selected (0.006 seconds)
set mapreduce.reduce.memory.mb;
±---------------------------------+
| set |
±---------------------------------+
| mapreduce.reduce.memory.mb=2048 |
±---------------------------------+
1 row selected (0.013 seconds)
set yarn.nodemanager.vmem-pmem-ratio;
±--------------------------------------+
| set |
±--------------------------------------+
| yarn.nodemanager.vmem-pmem-ratio=2.1 |
意味着节点管理器可以使用的虚拟内存是其可用物理内存的2.1倍！
±--------------------------------------+
1 row selected (0.006 seconds)
set yarn.nodemanager.vmem-check-enabled;
±-------------------------------------------+
| set |
±-------------------------------------------+
| yarn.nodemanager.vmem-check-enabled=false |
±-------------------------------------------+
1 row selected (0.005 seconds)
set dfs.block.size;
±--------------------------+
| set |
±--------------------------+
| dfs.block.size=134217728 |
±--------------------------+
1 row selected (0.007 seconds)
set hive.exec.reducers.bytes.per.reducer;
±-----------------------------------------------+
| set |
±-----------------------------------------------+
| hive.exec.reducers.bytes.per.reducer=67108864 |
±-----------------------------------------------+
1 row selected (0.007 seconds)
set mapreduce.map.java.opts;
±---------------------------------------------------+
| set |
±---------------------------------------------------+
| mapreduce.map.java.opts=-Djava.net.preferIPv4Stack=true |
±---------------------------------------------------+
1 row selected (0.005 seconds)
set mapreduce.reduce.java.opts;
±---------------------------------------------------+
| set |
±---------------------------------------------------+
| mapreduce.reduce.java.opts=-Djava.net.preferIPv4Stack=true |
±---------------------------------------------------+
1 row selected (0.006 seconds)
1、增大内存
set mapreduce.map.memory.mb=8192;
set mapreduce.reduce.memory.mb=4096;
set mapreduce.map.java.opts=-Xmx6144m;
set mapreduce.reduce.java.opts=-Xmx3072m;
2、减小单批次处理数据量