Which shell are you using? Bash, HP-UX POSIX /usr/bin/sh, or something different?
The shell might be running with job control disabled. In any POSIX-compliant shell, "echo $-" should produce a string of letters (and possibly numbers). If that string includes lowercase letter 'm', job control is enabled. Interactive shells normally run with job control enabled by default, but I guess some login script might disable it with "set +m". Unless you're running a restricted shell, this setting should be changeable by the user.
Run "stty -a |grep tostop". If the "tostop" TTY option is set, any background job stops as soon as it tries to produce any output to the terminal. If the result includes "-tostop", the tostop output is not set.
When you use nohup, is it really executing /usr/bin/nohup or something else (maybe a shell internal command or an alias)?
"which nohup" or "whence nohup" might be useful to confirm you're using the real nohup command.
Does the java program try to do something special with the TTY input/output? For example, if it's trying to request a password, it might use /dev/tty explicitly to bypass any input/output redirection, much like the "passwd" command does.
MK
MK