For a more qulaified, less frustrating and quicker approach but maybe at a $$ cost. You are better off paying to have someone just come in and do it or calling 3Com support. However, if like most these are the options of last instead of first resort, then read ahead.
Short answer for the DIY folks who like to read and figure it out themselves is to research the 3Com Knolwedgebase and your trusty friend Google. I just checked and 3Com does have a knowledge base in which option 184 info is probably there as is but configuring "Tribox" specifically, may not, but Asterisk maybe.
For those Gen X ers who prefer forums, texting etc read on.
Lets answer you 2nd question first.
It depends on you network design of course but from a high level. When any device sends a DHCP request. The device gets, uses and will continue to request to use, the same or a new IP address, for the subnet configured in the scope of the DHCP server that responded first. Of course there are a variety of techniques than can be applied to mitigate this and make it more predictable.
For 3Com phone its DHCP Option 184 which gives this flexability. among many things, Option 184 is used to make a phone "learn" a specfic VLAN ID. Once equipped with a VLAN ID the phone will then send a 2nd DHCP request using this VLAN ID and get an IP address from the DHCP server on/for the subnet associated with that VLAN ID. Since a single VLAN ID is typically used (there are exceptions of course) across the network for all "Voice" traffic. Configure both DHCP Servers/scopes with the same Option 184 info and in most cases this works fine.
For your first question
You are playing at the fringes between products based on an open source application and traditional (call it what you will) PBX's in VoIP Telephony
Basically "Tribox" an Asterisk based PBX. Regardless when configured for SIP and assuming UDP. It must be sending packets with port number in the area of 5060. Assuming the packets are getting to the other PBX (NBX). Most SIP PBX's have 2 options to allow this connection.
1 It must register = login and be authenticated usually using an extension. These are typically but not exclusively devices without a fixed IP address i.e. Phones using DHCP
2 Be a trusted IP address = No authentication needed since the Admin(you) has specifically configure this IP address on the PBX. Typically these are other VoIP PBX's, peripheral applications i.e Call Center App, T1 Gateways etc
