- Community Home
- >
- Software
- >
- HPE Ezmeral Software platform
- >
- configure.sh with multiple history servers, histor...
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Forums
Discussions
Discussions
Discussions
Forums
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО11-17-2023 01:17 PM - last edited on тАО11-20-2023 08:00 PM by support_s
тАО11-17-2023 01:17 PM - last edited on тАО11-20-2023 08:00 PM by support_s
configure.sh with multiple history servers, history server fails
Hi All,
I'm setting up a new cluster and when I run configure.sh i'm giving both history servers like below.
The problem is when i try to run.a haddop job it's giving the following error:
2023-11-17 14:08:00,130 ERROR org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer: Error starting JobHistoryServerjava.lang.IllegalArgumentException: Does not contain a valid host:port authority: host0001470.acme.com,host0001476.acme.com:10020 (configuration property mapreduce.jobhistory.address)
History server shuts down at this point.
CONFIGURE COMMAND:
/opt/mapr/server/configure.sh -secure -u mapradmin -g mapradmin -Z host0001467.acme.com:1181,host0001470.acme.com:1181,host0001473.acme.com:1181 -C host0001474.acme.com:7222,host0001475.acme.com:7222,host0001476.acme.com:7222 -H 7443 -N MAPRPOC -dare -HS host0001470.acme.com,host0001476.acme.com --isvm -D /dev/sdb,/dev/sdc -d host0001477.acme.com:3306 -dp password -du user -no-autostart
The resultant lines in mapred-site.xml look like this:
<property>
<name>mapreduce.jobhistory.address</name>
<value>host0001470.acme.com,host0001476.acme.com:10020</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>host0001470.acme.com,host0001476.acme.com:19888</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.https.address</name>
<value>host0001470.acme.com,host0001476.acme.com:19890</value>
</property>
My question is should i only be providing one history server in the configure.sh command ?
How is the HA functionality handled if I only provide one history server in the configure.sh command?
Thanks!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО11-20-2023 11:06 AM
тАО11-20-2023 11:06 AM
Re: configure.sh with multiple history servers, history server fails
The configure.sh script is designed to only handle a single HS node. Looking at the script itself:
# History Server IP hs_ip=
...
-HS) shift hs_ip="$1" ;;
That's why the mapred-site.xml file lists the string you passed to "-HS" and appends the port number rather than have the port number appended to each hostanme individually.
As for how high availability is handled with the HistoryServer, I'll leave that for others with more knowledge of the history server to recommend.
I work at HPE
HPE Support Center offers support for your HPE services and products when and how you need it. Get started with HPE Support Center today.
[Any personal opinions expressed are mine, and not official statements on behalf of Hewlett Packard Enterprise]

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО11-22-2023 11:20 AM
тАО11-22-2023 11:20 AM
Re: configure.sh with multiple history servers, history server fails
Hi Dave,
I modiffied the mapred-site.xml to include history server properties with a ":<port_number> after each node and still seeing the error (pasted below):
I'm wondering if this is something that is not possible?
The new configuratio in mapred-site.xml:
<property>
<name>mapreduce.jobhistory.address</name>
<value>host003017.acme.com:10020,host003019.acme.com:10020</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>host003017.acme.com:19888,host003019.acme.com:19888</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.https.address</name>
<value>host003017.acme.com:19890,host003019.acme.com:19890</value>
</property>
2023-11-22 12:12:42,956 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer: registered UNIX signal handlers for [TERM, HUP, INT]
2023-11-22 12:12:43,419 INFO org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer failed in state INITED
java.lang.IllegalArgumentException: Does not contain a valid host:port authority: host003017.acme.com:10020,host003019.acme.com:10020 (configuration property mapreduce.jobhistory.address)
hat org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:242)
hat org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:198)
hat org.apache.hadoop.yarn.conf.YarnConfiguration.getSocketAddr(YarnConfiguration.java:4779)
hat org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.getBindAddress(JobHistoryServer.java:198)
hat org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.doSecureLogin(JobHistoryServer.java:186)
hat org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.serviceInit(JobHistoryServer.java:127)
hat org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
hat org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.launchJobHistoryServer(JobHistoryServer.java:231)
hat org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.main(JobHistoryServer.java:241)
2023-11-22 12:12:43,435 ERROR org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer: Error starting JobHistoryServer
java.lang.IllegalArgumentException: Does not contain a valid host:port authority: host003017.acme.com:10020,host003019.acme.com:10020 (configuration property mapreduce.jobhistory.address)
hat org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:242)
hat org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:198)
hat org.apache.hadoop.yarn.conf.YarnConfiguration.getSocketAddr(YarnConfiguration.java:4779)
hat org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.getBindAddress(JobHistoryServer.java:198)
hat org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.doSecureLogin(JobHistoryServer.java:186)
hat org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.serviceInit(JobHistoryServer.java:127)
hat org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
hat org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.launchJobHistoryServer(JobHistoryServer.java:231)
hat org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.main(JobHistoryServer.java:241)
2023-11-22 12:12:43,437 INFO org.apache.hadoop.util.ExitUtil: Exiting with status -1: Error starting JobHistoryServer
2023-11-22 12:12:43,442 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down JobHistoryServer at host003017.acme.com/172.16.23.139
Thanks in advance.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО11-22-2023 11:26 AM
тАО11-22-2023 11:26 AM
Re: configure.sh with multiple history servers, history server fails
The point I was making is the configure.sh script was only expecting a single HS node. Does everything work properly when you modify mapred-site.xml and only list a single HS node?
I work at HPE
HPE Support Center offers support for your HPE services and products when and how you need it. Get started with HPE Support Center today.
[Any personal opinions expressed are mine, and not official statements on behalf of Hewlett Packard Enterprise]

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО11-22-2023 11:50 AM
тАО11-22-2023 11:50 AM
Re: configure.sh with multiple history servers, history server fails
Hi Dave,
Single host listed for __HS_IP__ then yes, everything works.
I was just exploring the HA compatibility as this format is used in some other files.
Thanks.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО11-22-2023 11:53 AM
тАО11-22-2023 11:53 AM
Re: configure.sh with multiple history servers, history server fails
Cool, just making sure things work when configured with a single HS node. I'll let others with more experience with HS weigh in on the appropriate way to offer high availability for this service.
I work at HPE
HPE Support Center offers support for your HPE services and products when and how you need it. Get started with HPE Support Center today.
[Any personal opinions expressed are mine, and not official statements on behalf of Hewlett Packard Enterprise]
