topic Re: Third party IPAM/DNS integration in HPE Morpheus Enterprise

Third party IPAM/DNS integration

Fri, 21 Jul 2023 14:35:26 GMT

I have an existing self-service VM build that currently uses built-in Morpheus IPAM and the MS DNS plug-in. I’m going to need to move this over to a third party tool that, unfortunately, does not have a plug-in currently available.

I have what I need to create tasks that call the IPAM/DNS service to do the individual steps needed to determine/reserve/delete IP addresses. What’s less obvious is how to present this in the catalog and apply it at provisioning time, as the documentation seems to lean toward using DHCP, IP Pools, or having a user-provided IP.

What I’m trying to do is more have the user select a network (or limit it to 1 based on environment selected), have that network be used as an input sent to the IPAM solution to get the next available IP, then use that IP to provision the VM. Is there a preferred/standard way to do something like this?

Re: Third party IPAM/DNS integration

Thu, 27 Jul 2023 15:05:46 GMT

That’s what I thought; the CIDR field is correct on the network. However, the subnet is null on the spec that’s created. Do I need to generate the subnet myself? I had assumed if I gave the network ID, it’d pull in the details from that.

This environment’s still on v5.5.2

Re: Third party IPAM/DNS integration

Thu, 27 Jul 2023 16:59:40 GMT

Understood. Will see how soon I can get the upgrade done.

Re: Third party IPAM/DNS integration

cbunge — Tue, 15 Aug 2023 13:56:30 GMT

Hey @John quick question (sorry if answered in this thread). What IPAM are you using?

Re: Third party IPAM/DNS integration

cbunge — Sat, 22 Jul 2023 15:13:46 GMT

Note this is basically that script + the IP hash all in one. It’s a few years old so it may use some older methods or variables, but if you needed to see it in full glory!

<#
    .SYNOPSIS
        Modify Payload of Initial Provision

    .Notes

        This can be manipulated to modify the submit payload and pre-tasks anyway you see fit.
        It will be typical for the submit to pause while these tasks execute as we are confirming the final payload.

    .DESCRIPTION
        The steps are as follows:
        1. Query Morpheus Pool based on Selection, set IP reserved
        2. Pass IP to Name Hashing
        3. Inject Instance Name and Host Name as Hashed Name
        4. Profit

    .PARAMETER bearer
    Specifies a Bearer token for Morpheus Automation

    .PARAMETER morphUrl
    Specifies the Morpheus URL in format 'https://url.com/'

    .PARAMETER configJson
    This is the incoming 'spec' JSON that holds the initial payload.  This is needed to know the pool ID.
    You will always pass the 'spec' variable in which is a system var and not a custom Option Type required.
    
    .EXAMPLE
        1. Local Shell Script Referencing a GIT Repo
        2. Command Example:
            pwsh -file 'path/script.ps1' -morphUrl '<%=morpheus.applianceUrl%>' -bearer "<%=cypher.read('secret/bearer')%>" 
                -configJson '<%=spec.encodeAsJson().toString()%>' -tags "<%=results.tags%>"
#>

#################################################################################
### Params
Param (
    [Parameter(Mandatory = $true)]$bearer,
    [Parameter(Mandatory = $true)]$morphUrl,
    [Parameter(Mandatory = $true)]$configJson,
    [Parameter(Mandatory = $true)]$tags
    #[AllowEmptyString()]$placeholder
)

### Functions
function ipv4ToHexString($ipv4){
 
    # Validation
    $valid = $true
    $ipv4.split(".") | foreach { if([int]$_ -ge 0 -and [int]$_ -le 255) { } else { $valid = $false } }
 
    if($valid -eq $true){
 
        # Conversion
        $hexString = ""
        $hexList = @("0", "1", "2", "3", "4", "5", "6", "7", "8", "9", "a", "b", "c", "d", "e", "f")
        $ipv4.split(".") | foreach {
            $mod = [int]$_ % 16
            $first = $hexList[(([int]$_ - $mod) / 16)]
            $second = $hexList[$mod]
            $hexString += $first+$second
        } 
        return $hexString
    }
    else{
        return $false
    }
}

### Variables
$tags = $tags | convertfrom-json
$configJson = $configJson | convertfrom-json -Depth 10
$ContentType = 'application/json'
$morphHeader = @{
    "Authorization" = "BEARER $bearer"
}
$poolApi = 'api/networks/pools/'
$pool = $configJson.networkInterfaces.network.pool.id
$tempName = get-random
$poolBody = @"
{
    "createDns": false,
    "networkPoolIp": {
      "hostname": $tempName
    }
}
"@

#################################################################################
### Script
# Request IP
$ip = (Invoke-WebRequest -Method POST -Uri ($morphUrl + $poolApi + $pool + '/ips') -SkipCertificateCheck -Headers $morphHeader -Body $poolBody -ContentType $ContentType).content | convertfrom-json
$ip = $ip.networkPoolIp

# Generate Name Hash
$hashName = (ipv4ToHexString -ipv4 $ip.ipAddress).toUpper()
$hashName = "VN$hashName"

# Update JSON

$networkJson = $configJson.networkInterfaces

$configJson.instance.name = $hashName
$configJson.instance.hostName = $hashName
$configJson.customOptions.name = $hashName
$configJson.customOptions.hostName = $hashName
$networkJson | Add-Member NoteProperty -Name ipAddress -Value $ip.ipAddress -f
$networkJson | Add-Member NoteProperty -Name ipMode -Value 'static' -f
$networkJson | Add-Member NoteProperty -Name replaceHostRecord -Value $true -f
$configJson.networkInterfaces = $networkJson
$configJson.metadata = New-Object pscustomobject
$configJson.metadata = $tags

#################################################################################
### Export

$configJson = $configJson | ConvertTo-Json -Depth 10

$spec = @"
{
    "spec": $configJson
}
"@

$spec

Re: Third party IPAM/DNS integration

Tue, 15 Aug 2023 14:58:35 GMT

@cbunge, it’s Micetro from Men & Mice

Re: Third party IPAM/DNS integration

Fri, 21 Jul 2023 18:53:27 GMT

Thanks @cbunge ,

After looking through it a bit, I think what I need to do is:

Grab the spec
Extract the network ID
Use the network ID to call the Morpheus API to get network details
Extract from that the CIDR
Call the IPAM solution for the next free IP address on that network
Call the IPAM solution to reserve that IP

What’s not clear yet is what to do with the IP at this point. In the example, you’re replacing the existing name with the generated hash, but the instance and host names already exist. When I do a test build (using the built-in Morpheus IPAM that’s currently assigning an IP), the IP isn’t part of this json yet.

Would it be as simple as assigning the IP to instance.networkInterfaces[0].ipAddress or am I missing something?

Re: Third party IPAM/DNS integration

cbunge — Sat, 22 Jul 2023 15:06:05 GMT

Assuming your networks [in Morpheus] are configured as DHCP, but has the network details filled out. I.E - GW, CIDR, DNS. This could be broken up into tasks in the configuration phase. The important note, the LAST task in that phase must:

Have Result Type set to JSON
Must only echo out the spec JSON into stdout. No other output

Example script (may have a few typos but should get you in the right direction):

# Get Morpheus Vars
$spec = '<%=spec.encodeAsJson().toString()%>' | convertfrom-json
$morphUrl = '<%=morpheus.applianceUrl%>'
$morphToken = '<%=morpheus.apiAccessToken%>'
$networkId = $spec.instance.networkInterfaces[0].id
$headers = @{
    "Authorization" = "BEARER $morphToken"
    }
$ContentType = 'application/json'

# Make API call to Morpheus to get more network details (if required) 
$network = (invoke-webrequest -method get -uri ($morphUrl + "api/networks/" + $networkId) -contentType $contentType -headers $headers).content  | ConvertFrom-Json | select -ExpandProperty network

# Make API cll to your IPAM.  THIS MUST NOT OUTPUT STDOUT |  out-null
$mynewIP = insert cool code here # assuming returns only IP 

# Update the spec
$networkJson = $configJson.networkInterfaces
$networkJson | Add-Member NoteProperty -Name ipAddress -Value $mynewIP -f
$networkJson | Add-Member NoteProperty -Name ipMode -Value 'static' -f
$spec.networkInterfaces = $networkJson

$spec = $spec | ConvertTo-Json -Depth 10

$spec = @"
{
    "spec": $spec
}
"@

$spec

Get Network API endpoint will get you the data as you need for your API call

#spec #configurationphase

Re: Third party IPAM/DNS integration

cbunge — Thu, 27 Jul 2023 15:31:07 GMT

I believe it’s always null until the spec fully submits and does a lookup on the network id.

Also, you are actually on the release the static IP issue was introduced :-/ 5.5.2 introduced IPv6 which caused an issue statically assigned IPv4 addresses when not polled from an IPAM. We have 2 newer LTS branches (you are currently on a Standard Branch). Is it possible to upgrade to either 6.0.5 or 6.2.0 to resolve?

Re: Third party IPAM/DNS integration

Mon, 31 Jul 2023 23:43:53 GMT

Upgrading seemed to do the trick based on initial testing. I reserve the right to necro this thread at a later time, but seems like we might be in business.

As always, thanks for the support

Re: Third party IPAM/DNS integration

Thu, 27 Jul 2023 14:13:05 GMT

Thanks @cbunge. Sorry for the delay between responses; I’ve been fighting a cold and progress is slow-going.

This is close to what I already had, conceptually, up until sticking the IP into the networkInterfaces. Anyway, I’m almost there. The IP is being selected, returned, and shows up on a newly provisioned instance in Morpheus. However, it’s not getting assigned on the actual VM itself.

I’m testing with a network that was originally configured with a Morpheus-owned IP pool, so I’m going to spend a while double checking all the settings there and see if something’s not entered incorrectly.

A couple things I did have questions about from your earlier replies:

You mentioned having the networks configured as DHCP in Morpheus. Is this necessary, or could they be static without an IP pool assigned? I’m curious what the DHCP setting does behind the scenes, as I’d expect it might leave the VM set to automatically obtain the network settings (e.g., gateway, dns servers, etc.) rather than set them based on the network’s settings.
The more complete script you provided also sets the property replaceHostRecord which I don’t see anywhere else. What’s he do?

Re: Third party IPAM/DNS integration

cbunge — Thu, 27 Jul 2023 14:28:29 GMT

Hey John,

So my recommendation for the DHCP was simply because that way Morpheus instance provisioning wouldn’t ask your end user for an IP address up front. As long as you modify things in the Configuration Phase nothing is submitted to a cloud / IPAM / etc until that phase completes. My script does update the mode to static, however.

As for replaceHostRecord… that’s the only time in my life I’ve used that I believe it was specific to InfoBlox. Since I was setting a temp name, it was creating a DNS record so I had to tell InfoBlox to update the DNS record to the final hostname.

One thing I would look into on getting the IP set, make sure all of the network details are filled out [on the network itself] in Morpheus (GW,CIDR,DNS) you may be failing to set a static IP if we don’t have all of the details to fully set it within the OS.

Re: Third party IPAM/DNS integration

Thu, 27 Jul 2023 14:48:34 GMT

Cool. I’ll leave it static then. This will ultimately be behind a catalog item once I get it working, so I’ll abstract field asking for an IP away from the users.

I did find an error in the vCenter logs stating that the parameter spec.nicSettingMap.adapter.subnetMask is not correct. When I look at the network under Infrastructure > Networks > networkId, the Netmask appears blank. Am I right in assuming this is related? If so, where does this get pulled from…I can edit the network to change the CIDR, DNS servers, and gateway (which are all present/correct), but don’t see where the subnet mask would come from if it’s not getting it from the CIDR.

Re: Third party IPAM/DNS integration

cbunge — Thu, 27 Jul 2023 15:02:55 GMT

Actually a better question, what version are you on? I think 6.0.0/6.0.1 had an issue with static (non-pool) ip addressing

Re: Third party IPAM/DNS integration

cbunge — Fri, 21 Jul 2023 14:46:36 GMT

So if you are not looking to build an IPAM plugin (totally understand), the best path would be using the Configuration Phase in a provisioning workflow.

Take a look at this ongoing thread talking about doing a hostname replacement. Basically would be the same process (and I have an example of using a manual IPAM call of my own!

Re: Third party IPAM/DNS integration

cbunge — Thu, 27 Jul 2023 15:01:59 GMT

Subnet/Netmask is interpreted from the CIDR field. Make sure it is in the following format: