Hive heap size question

[prusyn]

Hello everyone,

I struggle to understand how do I increase the Hive heap memory. I tred to change HADOOP_HEAPSIZE but it looks like it doesn't reflect changes. 

How can I increase the Hive heap size? Does HADOOP_HEAPSIZE work for Hive?

Thanks in advance!

Pavlo.

[mchalaev · Accepted solution]

Hi @prusyn, 

No, "HADOOP_HEAPSIZE" does not work for Hive now. When "HADOOP_HEAPSIZE" is used, it is always overridden by 512 Mb specified in hadoop-env.sh. This issue described in HADOOP-10245  and fixed in HADOOP-9902 .

For increasing Hive heap size, you can use "HADOOP_CLIENT_OPTS" option. e.g.:

export HADOOP_CLIENT_OPTS="${HADOOP_CLIENT_OPTS} -Xmx4096m"

If you want to configure this value separately for each Hive service you can use:

if [ "$SERVICE" = "hiveserver2" ]; then #For hiveserver2
export HADOOP_CLIENT_OPTS="${HADOOP_CLIENT_OPTS} -Xmx4096m"
fi
if [ "$SERVICE" = "metastore" ]; then #For hive metastore
export HADOOP_CLIENT_OPTS="${HADOOP_CLIENT_OPTS} -Xmx1024m"
fi
if [ "$SERVICE" = "cli" ]; then #For hive shell
export HADOOP_CLIENT_OPTS="${HADOOP_CLIENT_OPTS} -Xmx1024m"
fi

Thanks,
Murshid  Chalaev